JP6594222B2 - Sound source information estimation apparatus, sound source information estimation method, and program - Google Patents

Description

  The present invention relates to the technical field of acoustic signal processing, and more particularly to a technique for estimating sound source information using sound source features extracted from an acoustic signal.

  Conventionally, in the technical field of acoustic signal processing, sound source information has been estimated from sound source characteristics. As such conventional techniques, for example, a direct ratio estimation technique for estimating a direct ratio that is an index such as the degree of reverberation in a room, or a sound source that emphasizes a sound source at a specific position in a situation where various noises are mixed There are emphasis techniques.

In the conventional direct ratio estimation technique, the direct ratio is determined deterministically using a spatial sensitivity distribution obtained from a plurality of beamforming outputs (see, for example, Patent Document 1). FIG. 1 shows a functional configuration of a conventional direct ratio estimation apparatus. Frequency domain transforming units 11-1 to 11-M (M ≧ 2) receive signals received by M microphones 10-1 to 10-M as input, and frequency domain observation signals _{xω, τ} = [X _{1, ω, τ 1,} ..., X _{M, ω, τ} ] ^T is output. The beam forming unit (for direct sound enhancement) 12-1 receives the observation signal _{xω, τ} and outputs an output signal _{YBF, 1, ω in} which the sound source direction is enhanced. The beamforming unit (for diffuse reverberation analysis) 12-2 receives the observation signal _{xω, τ} and outputs an output signal _{YBF, 2, ω} in which the direction other than the sound source is emphasized. Local PSD estimator 13, two beamforming output signal _{Y BF, 1, ω, Y} BF, 2, using the _omega, PSDP _D of the direct sound in each _{frequency, omega} and reverberation of PSDP ^R, _omega Ask for. Power ratio calculation unit 14, PSDP _D of the direct sound and the _{reverberation, ω,} ^- P _R, with _omega, estimates the Chokkan ratio gamma.

In the conventional sound source enhancement technique, (1) a method using linear beamforming and (2) a non-linear Wiener filter is generated using a difference in spatial sensitivity distribution obtained from a plurality of beamforming outputs. The method etc. are used (for example, refer patent document 2). FIG. 2 shows a functional configuration of a conventional sound source enhancement apparatus. The sound receiving units 10-1 to 10-M observe sound using a general-purpose microphone array or a mutual information increasing type sound receiving system. The frequency domain transforming units 11-1 to 11-M receive the received M signals and output observation signals _{xω, τ} in the frequency domain. The beam forming unit (for direct sound enhancement) 12-1 receives the observation signal x _{ω, τ} and outputs an output signal Y _{ζ (1), ω, τ} = Y _{i, ω, τ in} which the sound source direction is enhanced. To do. Similarly, beam forming units (for noise analysis) 12-2 to 12-L (L ≧ 2) receive observation signals x _{ω and τ} , and L−1 beam formings in which directions other than the sound source are emphasized. Output signals Y _{ζ (2), ω, τ} ,..., Y _{ζ (L), ω, τ} are output. The local PSD estimation unit 13 estimates local PSD ^ Φ _{S, ω, τ} using _L beamforming output signals Y _{ζ (1), ω, τ} ,..., Y _{ζ (L), ω, τ.} To do. The a priori SNR calculation unit 15 receives the local PSD ^ Φ _{S, ω, τ} , and obtains an estimated value of the prior SNR ^ ξ _{ω, τ} = ^ φ _{S, ω, τ} / ^ φ _{N, ω, τ} . The filtering unit 16 calculates a Wiener filter using the estimated prior SNR ^ ξω _{, τ} , and multiplies the beamforming output signal Y _{ζ (1), ω, τ} by the Wiener filter, thereby obtaining the output signal Z _i, Output _{ω, τ} .

JP 2011-53062 A International Publication No. 2015/129760

  In the conventional direct ratio estimation technique, if conditions such as (1) direct sound and reverberation are uncorrelated, (2) only a sound source exists in the room and the noise level is sufficiently low, the direct ratio is accurately estimated to some extent. The ratio can be estimated. However, there is a correlation between the direct sound and the reverberation including the initial reflection, and there are many cases where noise exists in the room. In that case, the estimation accuracy of the direct ratio decreases.

  In the conventional sound source enhancement technology, (1) the time sparsity of the sound source signal group is low, and no correlation between the sound sources cannot be assumed, (2) the background noise level is high, and (3) the parameters used for the Wiener filter design are In the case of mismatch, the estimation accuracy of the target sound or noise PSD may be lowered, and the output signal may not be emphasized.

  That is, the conventional method for determining sound source information deterministically has a problem that the accuracy is lowered when used in an environment that does not satisfy a specific condition.

  In view of these points, an object of the present invention is to provide sound source information estimation capable of estimating sound source information from sound source features with high accuracy even when the estimation accuracy of conventional deterministic techniques is reduced. Is to provide technology.

  In order to solve the above-described problem, the sound source information estimation device according to the present invention provides a sound source for each frequency domain observation signal from a plurality of frequency domain observation signals picked up by emphasizing sounds coming from a plurality of angular regions in different directions. A sound source feature extraction unit that extracts features, and a sound source information estimation unit that inputs a sound source feature of each frequency domain observation signal to a statistical mapping model to obtain an estimation value of the sound source information. Parameters were learned using sound source features extracted from multiple frequency domain acoustic signals picked up by emphasizing sounds coming from angular regions in different directions and correct values of sound source information obtained from each frequency domain acoustic signal Is.

  According to the sound source information estimation technique of the present invention, the sound source information is obtained from the sound source features by a statistical method. Therefore, even in an environment where the estimation accuracy is reduced by the conventional deterministic method, the sound source features are accurately obtained. From this, sound source information can be estimated.

FIG. 1 is a diagram illustrating a functional configuration of a conventional direct ratio estimation apparatus. FIG. 2 is a diagram illustrating a functional configuration of a conventional sound source enhancement device. FIG. 3 is a diagram for explaining a decomposition model of a sound source and an impulse response. FIG. 4 is a diagram for explaining a propagation model of direct sound and reverberation in a reverberant environment. FIG. 5 is a diagram for explaining local PSD estimation for estimating the direct ratio. FIG. 6 is a diagram illustrating a functional configuration of the direct ratio estimation apparatus according to the first embodiment. FIG. 7 is a diagram illustrating a processing procedure of the direct ratio estimation method. FIG. 8 is a diagram illustrating a functional configuration of the DNN mapping unit. FIG. 9 is a diagram illustrating a functional configuration of the direct ratio estimation apparatus according to the second embodiment. FIG. 10 is a diagram illustrating a functional configuration of the direct ratio estimation apparatus according to the third embodiment. FIG. 11 is a diagram illustrating a functional configuration of the direct ratio estimation apparatus according to the fourth embodiment. FIG. 12 is a diagram for explaining the mutual information increasing sound receiving system. FIG. 13 is a diagram illustrating a functional configuration of the sound source emphasizing device according to the fifth embodiment. FIG. 14 is a diagram illustrating a processing procedure of the sound source enhancement method. FIG. 15 is a diagram illustrating a functional configuration of the sound source emphasizing device according to the sixth embodiment. FIG. 16 is a diagram illustrating a functional configuration of the sound source emphasizing device according to the seventh embodiment. FIG. 17 is a diagram illustrating a functional configuration of the sound source emphasizing device according to the eighth embodiment. FIG. 18 is a diagram illustrating an array structure having symmetry.

  Hereinafter, embodiments for carrying out the present invention will be described in detail. In addition, the same number is attached | subjected to the component which has the same function in drawing, and duplication description is abbreviate | omitted.

  In the first to fourth embodiments, embodiments in which the sound source information estimation technique of the present invention is applied to the direct ratio estimation technique will be described. In the fifth to eighth embodiments, embodiments in which the sound source information estimation technique of the present invention is applied to a sound source enhancement technique will be described. In the ninth embodiment, a sound source information estimation technique extracted as a superordinate concept of the direct ratio estimation technique and the sound source enhancement technique will be described in association with each embodiment.

Note that the symbols “^”, “ ⁻ ”, etc. used in the text should be written immediately above the character that immediately follows, but are written immediately before the character due to restrictions on textual notation. In the mathematical expression, these symbols are described in their original positions, that is, directly above the characters.

<Direct ratio estimation technology>
First, a direct ratio estimation technique using a conventional deterministic technique will be described in detail, and then an embodiment of the direct ratio estimation technique to which the sound source information estimation technique of the present invention is applied will be described.

≪Sound propagation model in reverberant environment≫
The impulse response between a sound source and a microphone in a reverberant environment is composed of a direct sound, an early reflection, and a late reverberation. Here, for the sake of simple modeling, it is assumed that the reverberation includes the initial reflected sound and the rear reverberation. As shown in FIG. 3, the reverberation includes two elements, a direct sound and a reverberation. Assume that a response is constructed. If the characteristic of the impulse response at frequency ω (hereinafter referred to as transfer characteristic) is H _ω , the transfer characteristic is modeled by equation (1).

Here, H _{D, ω} represents a direct sound transfer characteristic, and H _{R, ω} represents a reverberation transfer characteristic.

  The transfer characteristics of direct sound and reverberation are very different. In general, the direct sound is spatially coherent, and the reverberation is diffusive and has low coherence. The direct sound propagates directly to the microphone as shown in FIG. Conversely, reverberation can be modeled to propagate with equal power from all directions, as shown in FIG. By paying attention to the difference in this propagation model, it is considered that the power of direct sound and reverberation can be separated and estimated.

  In the following description, three conditions are assumed. (1) The direction of arrival of the sound source is known. The direction of arrival of the sound source may be given manually, or may be estimated by a conventional method such as the beam forming method or the MUSIC method. (2) Assume that the direct sound and reverberation are uncorrelated. (3) In order to search the sensitivity distribution in each direction, observation is made with a microphone array composed of a plurality of microphones.

≪Array signal observation model≫
Let X _{m, ω, τ be} the observation signal in the m-th microphone. There are a total of M microphones, ω represents a frequency bin number, and τ represents a time frame number. Using equation (1), X _{m, ω, τ} is modeled by equation (2).

Here, S _{ω, τ} represents the spectrum of the sound source.

The transfer characteristic in equation (2) is further broken down into two elements. The first element is a transfer characteristic from the sound source to the reference position (for example, the center of the microphone array). The second element is a transfer characteristic from the reference position to each microphone. Assuming that the sound source signal arrives as a plane wave, the second element (transfer characteristic between the reference position and each microphone) can be approximated by a phase shift due to a delay difference generated between the microphones. Therefore, H _{D, m, ω} and H _{R, m, ω} are expressed by equations (3) and (4).

Here, H _{Dref, ω} represents a direct sound transfer characteristic between the sound source and the reference position, and H _{Rref, Ω, ω} represents a reverberation transfer characteristic. τ _{Ω, m} is the reference position when the sound source arrives from the three-dimensional angle Ω = {θ, φ} (θ∈ [0, 2π]: horizontal angle, φ∈ [0, π]: zenith angle) This represents the delay time between the m-th microphones. Ω _D represents the direction of arrival of the sound source.

  The observed signal vector of the microphone array is modeled by equation (5).

Here, aΩ _{, ω} = [exp (−jωτ1 _{, Ω} ),... Exp (−jωτM _{, Ω} )] ^T represents a steering vector with respect to the direction Ω. S _{D, ω, τ} , S _{R, Ω, ω, τ} represent direct sound and reverberation coming from the direction Ω observed at the reference position, respectively, and are defined by equations (6) and (7).・^T (superscript T) indicates transposition.

≪Beam forming output≫
In order to analyze the intensity distribution of wavefronts coming from different directions, it is assumed that two or more beamforming filters are multiplied with the observation signal _{xω, τ} . The output signal of the l-th beamforming is expressed by equation (8).

Here, · ^H (superscript H) represents conjugate transpose. w _{l, ω} are filter coefficients of the l-th beamforming, and are defined by Expression (9).

  The power spectrum density (Power Spectral Density: PSD) of the beamforming output is expressed by adding the direct sound weighted by the beamforming sensitivity characteristic and the PSD of the reverberation, and is expressed by the following equation.

Here, P _{D, ω} represents the PSD of the direct sound observed at the reference position, and P _{R, Ω, ω} represents the PSD of reverberation. P _{D, ω} and P _{R, Ω, ω} are expressed by equations (12) and (13).

Here, E [•] represents an expected value calculation with respect to time. G _{l, Ω, ω} represents the sensitivity of the l-th beamforming filter with respect to the direction _{Ω, and} is expressed by Expression (14).

In the derivation of equation (11), it is assumed that the direct sound and reverberation are uncorrelated. That is, E [S ^* _{D, ω, τ} S _{R, Ω, ω, τ} ] = 0. Here, ^* (superscript *) represents a complex conjugate.

  Since it is assumed that the reverberation propagates spatially (ie, the power coming from all directions is equal), the reverberation PSD can be modeled as constant for all directions Ω.

  Therefore, the PSD of the beamforming output in Equation (11) is Equation (16).

≪Local PSD estimation≫
As shown in FIG. 5, it is assumed that two beamforming (Beamformers) having different directivity characteristics are convoluted with the array observation signal. According to equation (16), the PSDs of the two beamforming outputs are expressed in matrix form as in equation (17).

Here, since the elements included in P _{BF, ω} and G _ω are known and can be calculated in advance, the PSD of direct sound and reverberation is estimated for each frame.

Here, ^ · represents the estimated value. Incidentally, ^ P _cmp, P _D constituting the _{omega, omega} and ^- P _{R, omega} since a positive value, when the result calculated is a negative value is flooring to 0. Estimated P _D, and _omega ^- P _R, with _omega, can be estimated Chokkan ratio gamma _conv according to equation (19).

In the present invention, in order to improve the estimation accuracy of Chokkan ^{_{ratio, P D, ω, - P}} R, as input source characterized _omega, introducing statistical mapping model that outputs Chokkan ratio gamma. In recent years, a deep neural network (DNN) is often used as one of the statistical mapping models. Therefore, a deep neural network is used here. Details of the deep neural network are described in Reference Document 1 below.
[Reference 1] Takayuki Okaya, “Deep Learning”, first edition, Kodansha Scientific, 2015

  The points when the present invention is applied to the direct ratio estimation are (1) a configuration in which the estimated local PSD and a plurality of beamforming output power groups are input to the deep neural network, and the direct ratio is output. (2) The initial value of the network parameter of the deep neural network is set in consideration of physical characteristics as in the conventional method.

[First embodiment]
As shown in FIG. 6, the direct ratio estimation apparatus according to the first embodiment includes M microphones 10-1 to 10 -M, M frequency domain transform units 11-1 to 11 -M, and two beams. A forming unit 12-1 to 12-2, a local PSD estimation unit 13, and a DNN mapping unit 20 are provided. The direct ratio estimation apparatus of the first embodiment is realized by the processing of each step described later by this direct ratio estimation apparatus.

  The direct ratio estimation device is configured, for example, by loading a special program into a known or dedicated computer having a central processing unit (CPU), a main memory (RAM), and the like. It is a special device. The direct ratio estimation apparatus executes each process under the control of the central processing unit, for example. The data input to the direct ratio estimation device and the data obtained in each process are stored in, for example, the main storage device, and the data stored in the main storage device is read out as necessary for other processing. Used. Moreover, at least a part of each processing unit of the direct ratio estimation apparatus may be configured by hardware such as an integrated circuit.

  With reference to FIG. 7, the processing procedure of the direct ratio estimation method of the first embodiment will be described.

In step S10, a microphone array including M microphones 10-1 to 10-M collects M observation signals x _m (n) (m = 1,..., M). Here, n represents the sample number of the discrete time signal. The observation signal x _m (n) is input to each of the frequency domain conversion units 11-1 to 11-M.

In step S11, the frequency domain transform unit 11-m (m = 1,..., M) sets each observation signal x _m (n) to a short time length (for example, about 256 samples when the sampling frequency is 16,000 Hz). And the discrete Fourier transform is performed in each frame, and the frequency domain observation signals X _{m, ω, τ} are output. Here, ω represents a frequency bin number, and τ represents a time frame number. The output signal groups X1 _{, ω, τ} , X2 _{, ω, τ} ,..., XM _{, ω, τ} of the frequency domain transform units 11-1 to 11-M are transmitted to the beam forming units 12-1 to 12-2. Each is entered.

In step S12, the beam forming units 12-1 to 12-2 output signal groups X1 _{, ω, τ} , X2 _{, ω, τ} , ..., XM of the frequency domain transform units 11-1 to 11- _{M. , ω, τ} are processed to collect sounds by emphasizing sounds coming from angular regions in different directions, and the results are output. The beam forming unit 12-1 is used for direct sound enhancement, and outputs an output signal Y _{BF, 1, ω} by enhancing sound arriving from a predetermined sound source direction. The beam forming unit 12-2 is used for analysis of diffuse reverberation, and outputs an output signal _{YBF, 2, ω} by emphasizing sound coming from directions other than the sound source direction. The output signal groups Y _{BF, 1, ω} , Y _{BF, 2, ω} of the beam forming units 12-1 to 12-2 are input to the local PSD estimation unit 13.

In step S13, the local PSD estimation unit 13 receives the output signal groups Y _{BF, 1, ω} , Y _{BF, 2, ω} of each of the beam forming units 12-1 to 12-2 as input, and according to the above equation (18), Estimate PSDP _{D, ω of} direct sound and PSD ^- P _{R, ω} of reverberation. Estimated direct sound and reverberation ^{_{PSDP D, ω, - P R}} , ω is input to DNN mapping unit 20.

In step S20, DNN mapping unit 20, PSDP _D of the direct sound and the reverberation output of the local PSD estimator _{13, ω,} ^- P _R, as input _omega, estimates of Chokkan ratio using the network parameter z gamma And output the result.

  Hereinafter, the process of the DNN mapping unit 20 will be described in detail. As shown in FIG. 8, the DNN mapping unit 20 is composed of an N-layer deep neural network. Here, N may be about 4-5. First, feature values are set in the input layer of the deep neural network.

Here, ω _Q represents the number of analysis FFT bins. At this time, a state of a frequency band in which a plurality of frequency bins are aggregated may be used. If the network parameter z includes Z ⁽²⁾ ,…, Z ^(N) , b ⁽²⁾ ,…, b ^(N) , the output of the deep neural network is calculated as Is calculated as follows.

Here, when the number of layers n-th layer is described as J _n,

It is. The activation function f ⁽ⁿ⁾ (•) is expressed by the sigmoid function (when n = 2,…, N-1) and the identity mapping function (when n = N), as shown in Equation (28). ).

The number of layers in the Nth layer is J _N = 1, and the estimated direct ratio Γ is expressed by Equation (29).

Hereinafter, the direct ratio estimated using q ⁽¹⁾ as an input and the network parameter z is denoted as Γ (q ⁽¹⁾ ; z).

  Deep neural network has been able to properly set the initial values of network parameters by properly pre-training multi-layer neural networks based on the study of Hinton et al.'S deep brief network (DBN). As a result, it has been used in various fields. In deep brief nets, restricted Boltzmann machines (RBM) are stacked in multiple layers (stacked RBMs), and the initial values of network parameters are estimated for each layer. In order to appropriately calculate the update amount of the network parameter of each constrained Boltzmann machine, for example, a constructive divergence method (contrastive divergence: CD) may be used.

  Hereinafter, the optimization method of the deep neural network will be described. In the first embodiment, the initial value of the network parameter z is set at random, and the network parameter z is optimized so as to minimize the estimation error of the direct ratio based on back propagation. Teacher information consisting of a local PSD for learning composed of K pieces of sample data and the correct value of the direct ratio is described as in equation (30).

  When each step of Equations (21) and (22) is applied to K sample data, it can be written in matrix form as follows.

Here, b ⁽ⁿ⁾ 1 _K ^T represents an operation of arranging b ⁽ⁿ⁾ by K pieces,

It is.

  The square error function defined by Equation (35) is used as a measure for measuring the error between the output of the deep neural network and the direct ratio given as the correct answer.

Based on the back propagation error, the gradient of the network parameter is calculated sequentially from the output layer (n = N) to the input layer (n = 1). _{^{- Γ = [- Γ 1,}} ..., - Γ K] When the obtained delta in each sample data in the n th layer Δ a ⁽ⁿ⁾ as follows.

Where f '(•) is the derivative of the function f (•),

Represents the product of each component of the matrix. The slope of the error function is calculated as follows:

  Finally, the parameters are updated based on the obtained gradient.

The update amounts ΔZ ⁽ⁿ⁾ and Δb ⁽ⁿ⁾ may be set as follows.

Where ΔZ ^{(n) +} , Δb ^{(n) +} is the previous update amount, ε is a learning coefficient, μ is a momentum coefficient to improve generalization performance and speed up learning, and λ is weight attenuation (Weight decay). ε may be set to about 0.01, μ may be set to about 0.9, and λ may be set to about 0.0002.

[Second Embodiment]
In the first embodiment, PSDP _D of the direct _{sound, omega} and reverberation of PSDP ^R, as input _omega, has been described a structure using a deep neural network N layer that outputs the estimated value Γ of Chokkan ratio. In the second embodiment, a configuration using an N-layer deep neural network that outputs a plurality of beamforming outputs as inputs and outputs an estimate value Γ of the direct ratio will be described.

  As illustrated in FIG. 9, the direct ratio estimation apparatus according to the second embodiment includes M microphones 10-1 to 10 -M, M frequency domain transform units 11-1 to 11 -M, and two beams. Forming units 12-1 to 12-2 are provided in the same manner as in the first embodiment, and a DNN mapping unit 21 is further included. The local PSD estimation unit 13 provided in the direct ratio estimation apparatus of the first embodiment is not provided, and the outputs of the beamforming units 12-1 to 12-2 are input to the DNN mapping unit 21. The The direct ratio estimation apparatus of the second embodiment is realized by the processing of each step described later by this direct ratio estimation apparatus.

In step S21, the DNN mapping unit 21 receives the output signal groups Y _{BF, 1, ω} , Y _{BF, 2, ω} of each of the beam forming units 12-1 to 12-2 as input, and directly uses the network parameter z. Find the ratio estimate Γ and output the result. The DNN mapping unit 21 performs optimization in the same manner as in the first embodiment, using teacher information that is composed of K pieces of sample data and includes the beamforming output for learning and the correct value of the direct ratio. Is.

[Third embodiment]
In the first embodiment and the second embodiment, the initial value of the network parameter z is set at random. In the third embodiment, a method for setting the initial value of the network parameter z in consideration of physical characteristics as in the conventional method will be described. The direct ratio estimation technique in the conventional method is mainly composed of the following three steps.

As in: (Step 1 local PSD estimation process) formula (18), the output power group P _BF of two or more beamforming _estimate of the local PSD from _omega ^ P _cmp, seeking _omega.

(Step 2: Frequency addition processing) As shown in equation (19), by adding the estimated value of local PSD ^ P _{cmp, ω} over the entire frequency band, Σ _ω P _{D, ω} , Σ _ω P _{R, ω} Output.

As in: (Step 3 log domain ratio calculation process) formula _{(19), Σ ω P D} , ω, Σ ω P R, and outputs the _omega as follows estimates ^ gamma of Chokkan ratio.

  Since the above three-step processing can be regarded as physically corresponding to the processing of each layer, it is possible to give an initial value of a network parameter with higher quality than that set at random. In the third embodiment, the direct ratio estimation apparatus of the second embodiment is configured to set an initial value of the network parameter z. The optimization process is the same as in the second embodiment.

  As shown in FIG. 10, the direct ratio estimation apparatus according to the third embodiment includes M microphones 10-1 to 10-M, M frequency domain transform units 11-1 to 11-M, and two beams. The forming units 12-1 to 12-2 and the DNN mapping unit 21 are provided in the same manner as in the second embodiment, and further an initial value setting unit 31 is provided. The direct ratio estimation apparatus of the third embodiment is realized by the processing of each step described later by this direct ratio estimation apparatus.

  The initial value setting unit 31 sets initial values of network parameters corresponding to each layer of the DNN mapping unit 21 as follows. The value of the input layer is set as in the following equation, as in the second embodiment.

The processing of the second layer corresponds to (Step 1: Local PSD estimation processing). The number of layers in the second layer should be J ₂ ≧ L × Q. L is the number of beamforming, and Q is the number of frequency bins. Below, it demonstrates as L = 2. The local PSD estimation process can be expressed by writing the network parameters as follows.

G ₂ and B ₂ are price range adjustment coefficients. Set G ₂ so that the maximum value of Z ⁽²⁾ q ⁽¹⁾ is about 1-5. Also, B ₂ is set between -5 and 0 in order to floor the value near 0 when the output value of Z ⁽²⁾ q ⁽¹⁾ is 0 or less. Thereafter, the output q ⁽²⁾ of the second layer is obtained by performing the following calculation.

The processing of the third layer corresponds to (Step 2: Frequency addition processing). The frequency addition process can be expressed by writing the network parameters as follows. Note that the number of layers in the third layer is set to satisfy J ₃ ≧ 2.

G ₃ and B ₃ are price range adjustment coefficients. Set G ₃ so that the maximum value of Z ⁽³⁾ q ⁽²⁾ is about 1-5. In addition, B ₃ is about 0, and there is no problem. Thereafter, by calculating the equations (52) and (53), the output q ⁽³⁾ of the third layer is obtained.

The processing of the fourth layer corresponds to (Step 3: Logarithmic area ratio calculation processing). Logarithmic domain ratio calculation processing can be expressed by writing network parameters as follows. Note that the number of layers in the fourth layer (output layer) is J ₄ = 1.

Here, in order to correspond to the equation (44), Z _1,1 ⁽⁴⁾ is limited to a positive value, and Z _1,2 ⁽⁴⁾ is limited to a negative value. For example, the value is determined as follows.

At this time, the 10log ₁₀ (Σ _ω P _{D, ω} ) and ₁₀ log ₁₀ (Σ _ω P _{R, ω} ) being referred to may be calculated using one sample, or may be used for many samples. An average value of the calculated values may be used.

  Finally, the output value is calculated as follows.

  In the network parameter initial value setting method described above, the number of layers is preferably set to be equal to or greater than the minimum unit number of signal processing operations + 1. In the above description, it is assumed that N = 4. However, if N is desired to be greater than 4, a redundant layer may be interposed. Here, the minimum number of units of signal processing operations is equivalent to signal processing operations (here, direct ratio estimation processing), such as addition and multiplication, which are required when executing the conventional deterministic method. It means the number of signal processing operations.

[Fourth embodiment]
In the third embodiment, the configuration in which the initial value of the network parameter z is set in the direct ratio estimation apparatus of the second embodiment has been described. In the fourth embodiment, the direct ratio estimation apparatus of the first embodiment is configured to set an initial value of the network parameter z. The optimization process is the same as in the first embodiment.

  As shown in FIG. 11, the direct ratio estimation apparatus of the fourth embodiment includes M microphones 10-1 to 10-M, M frequency domain transform units 11-1 to 11-M, and two beams. The forming units 12-1 to 12-2, the local PSD estimation unit 13, and the DNN mapping unit 20 are provided as in the first embodiment, and further an initial value setting unit 30 is provided. The direct ratio estimation apparatus of the fourth embodiment is realized by the processing of each step described later by this direct ratio estimation apparatus.

  In the third embodiment, it has been described that the process of the second layer corresponds to the local PSD estimation process. Therefore, in the fourth embodiment, the initial value setting for the third and subsequent layers may be used. The value of the input layer is set as shown in Expression (60), as in the first embodiment.

  The processing after the equation (50) can be set as the initial values for the second and third layers. In the third embodiment, it has been described that it is better to set N ≧ 4. However, since the idea of setting the number of layers to be equal to or greater than the minimum unit number +1 of the signal processing operation is the same, in the fourth embodiment N It is desirable to set ≧ 3.

  As in the third embodiment and the fourth embodiment, by appropriately setting the initial values of the network parameters, the parameters can be optimized while maintaining the physical meaning of each layer. As a result, an effect of reducing the possibility of outputting an outlier even if the learning data is small to some extent is expected. That is, there is an effect that it is possible to design a deep neural network that is less dependent on the environment even if there is little learning data.

<Sound source enhancement technology>
First, a sound source enhancement technique using a conventional deterministic technique will be described in detail, and then an embodiment of a sound source enhancement technique to which the sound source information estimation technique of the present invention is applied will be described.

≪Modeling of observation signal≫
There are K sound sources in the sound field, and M (≧ 2) microphones are used for observation. This situation can be regarded as one of multiple-inputs and multiple-outputs (MIMO). Assuming that the transfer characteristic between the kth sound source and the mth microphone is A _{m, k, ω} , M observation signals _{xω, τ} can be calculated as shown in Equation (61).

Here, the equation (61) includes the following elements.

Here, the kth sound source is described as S _{k, ω, τ} , and the non-directional background noise in the _mth microphone is described as N _{m, ω, τ} . It is also assumed that the average value of the sound source and background noise and the expected value of power satisfy the following.

Here, <•> represents an expected value operator. Assuming that S _{k, ω, τ} and N _{m, ω, τ} are uncorrelated with each other, the following results.

Here, ^* (superscript *) represents a complex conjugate. When the above statistical properties are satisfied, the variance covariance matrix of the sound source signal and background noise is modeled as follows.

Here, · ^H (superscript H) is a conjugate transpose, I _K is a K-dimensional unit matrix, and I _M is an M-dimensional unit matrix.

The variance-covariance matrix (hereinafter referred to as the spatial correlation matrix) of the observation signals x _{ω, τ} is modeled below.

Here, R _{A, ω} is composed of received sound power σ ² _{A, ω} (assuming that the channel level is normalized in advance) and inter-channel correlations Γ _{i, j, ω} .

Hereinafter, the sound receiving system design technology, beam forming, and Wiener filtering that constitute the conventional sound source enhancement technology will be described in order. The sound receiving system design technique is a sound receiving technique (hardware) for analyzing a target sound source group in detail. Beam forming is a signal processing technique for processing received observation signal groups. Wiener filtering is a technique for performing further noise suppression on a signal after beamforming. It is a conventional technique to mount these techniques in any combination as described below.
Mounting form 1: Sound reception design technology + beam forming + Wiener filtering Mounting form 2: Sound receiving design technique + beam forming Mounting form 3: (General-purpose microphone) + Beam forming Mounting form 4: (General-purpose microphone) + Beam forming + Wiener filtering

≪ Mutual information increasing type receiving system design technology ≫
Reference 2 describes (1) the nature of a received signal that makes it easy to pick up a sound source signal and (2) a sound receiving system that uses a multi-concave reflector as one implementation. ing.
[Reference 2] K. Niwa, T. Kako, and K. Kobayashi, “Microphone array for increasing mutual information between sound sources and observation signals,” ICASSP 2015, pp. 534-538, 2015.

In the technique described in Reference 2, in order to measure how much information x _{ω, τ} teaches s _{ω, τ} to be analyzed in detail, the mutual information amount of s _{ω, τ} and x _{ω, τ} Define I _{s; x} .

Here, H _s represents the entropy of the transmission information amount, and H _{s | x} represents the transmission loss. If A _ω is not a regular matrix or if the background noise level is high, the transmission loss H _{s | x} increases. In order to investigate a spatial correlation matrix that maximizes I _{s; x} , a channel capacity C _ω is introduced.

By performing eigenvalue decomposition on R _{A, ω} , C _ω is expressed as follows.

Here, Λ _{m, ω} is the m-th eigenvalue of R _{A, ω} . According to reference 2, C _omega is maximized by received sound signals so that the eigenvalue distribution as follows are smoothed.

Receiving sound so that the eigenvalue is smoothed as in equation (82) corresponds to receiving sound so that the correlation between channels becomes zero.

If I _{s; x} increases, the observation signal group should include a clue for separating the sound source.

As a sound receiving system for increasing the mutual information amount I _{s; x} , (1) a diffuse sound receiving system (see the following reference 3), or (2) a sound receiving system using a multi-concave reflector (the above reference) 2). The diffuse sound receiving system is an array in which multiple reflection plates are installed so as to surround multiple microphones, utilizing a physical phenomenon in which the correlation between channels is lowered by discretely arranging microphones in a diffusion field. A sound receiving system using a multi-concave reflector is shown in FIG. Multiple microphones are installed near the focal point of the parabolic reflector. Near the focal point, the sound waves reflected by the parabolic reflectors arrive in various directions and time differences. By installing the microphone at a position slightly deviated from the focal position, the amplitude and phase of the received sound change dramatically. Therefore, if the position of the microphone is optimally set, the mutual information amount I _{s; x} increases. In the sound receiving system of FIG. 12, eight microphones are sub-optimally installed in front of each of the twelve parabolic reflectors so that mutual information increases, and a total of M = 96 omnidirectional microphones are mounted. Has been.
[Reference 3] K. Niwa, Y. Hioka, K. Furuya, and Y. Haneda, “Diffused sensing for sharp directive beamforming,” IEEE Trans. On Audio, Speech and Language Proc., Vol. 21, pp. 2346 -2355, 2013.

≪Sound source enhancement method 1: beam forming≫
A sound source enhancement method based on beamforming will be described. Beamforming is a method of enhancing a sound source coming from a specific direction by manipulating and adding the phase / amplitude difference generated between microphones. The output signal Y _{i, ω, τ} is obtained by multiplying the observation signal group _{xω, τ} by a filter w _{i, ω} that enhances the sound source coming from the i-th direction.

here,

It is.

Typical filter design methods include the delay sum method and the minimum variance method, which will be described below. First, the relationship between the phase / amplitude difference between microphones when a sound wave coming from the i-th direction is received is modeled. Hereinafter, it is referred to as a steering vector h _{i, ω} .

When a general-purpose microphone array (a non-directional microphone is disposed in a hollow space) is used and the distance between the sound source and the microphone is separated (for example, 1 meter or more), the steering vector can be modeled as follows.

Where c is the speed of sound (approximately 340 meters per second), p _i = [p _{X, i} , p _{Y, i} , p _{Z, i} ] ^T is the position vector of the i th sound source, and p _m = [p _{X, m} , p _{Y, m} , p _{Z, m} ] ^T represents the position vector of the m-th microphone. When a mutual information increasing type microphone array is used, transfer characteristics are used as a steering vector.

However, the measured impulse response includes room reverberation and tends to be long. Therefore, data obtained by cutting out a short section after the direct wave arrives may be used, or data calculated by simulation may be used.

  The delay sum filter is calculated by calculating the equation (89) using the steering.

When designing a filter by the minimum variance method, formula (90) is calculated.

Here, R _{H, ω} is a spatial correlation matrix designed using steering.

here,

It is. The output signal after beam forming in the time domain is obtained by _performing a short-time inverse Fourier transform on Y _{i, ω, τ} .

≪Sound source enhancement method 2: Wiener filtering based on local PSD estimation≫
In order to perform noise suppression with higher accuracy, a method of multiplying the beamforming output signals Y _{i, ω, τ} by a Wiener filter will be described. When the Wiener filter for enhancing the i-th sound source is Gi _{, ω, τ} , the output signal Z _{i, ω, τ} is obtained by Expression (93).

G _{i, ω, τ} is an amount that changes from frame to frame, and is calculated by equation (94).

Here, ^ φ _{S, ω, τ} represents the estimated value of the PSD of the target sound included in the signal after beamforming, and ^ φ _{N, ω, τ} represents the estimated value of the PSD of noise. ^ Ξ _{ω, τ} = ^ φ _{S, ω, τ} / ^ φ _{N, ω, τ} is an estimated value of the signal-noise ratio (hereinafter referred to as prior SNR) in the signal after beam forming. Represents. In either case, in order to design a Wiener filter, it is necessary to obtain the observation signal group x _{ω, τ} .

As a conventional method for obtaining the PSD of the target sound and noise from the observation signal group, there is a local PSD estimation method (see References 4 and 5 below).
[Reference 4] Y. Hioka, K. Furuya, K. Kobayashi, K. Niwa, and Y. Haneda, “Underdetermined sound source separation using power spectrum density estimated by combination of directivity gain,” in Proc. IEEE Trans. On Audio, Speech, and Language Proc., Vol. 21, pp. 1240-1250, 2013.
[Reference 5] K. Niwa, Y. Hioka, and K. Kobayashi, “Post-filter design for speech enhancement in various noisy environments,” in Proc. IWAENC 2014, pp. 36-40, 2014.

As described above, by applying beam forming to the observation signals x _{ω and τ} , it is possible to obtain a signal collected by enhancing a sound source that has arrived from a specific direction or position. In order to analyze not only the target sound but also noise information and estimate the PSD of the target sound and noise, L (≧ 2) beamformings are used. Suppose that the l (= 1,…, L) th beamforming emphasizes the sound source at the ζ (l) th position and picks up the sound, and the lth beamforming signal is expressed as Y _{ζ (l), ω, τ} . To express. A plurality of beamforming output signal groups are represented as y _{ω, τ} = [Y _{ζ (1), ω, τ} ,..., Y _{ζ (L), ω, τ} ] ^T. It is assumed that ζ (1) = i and that the first beamforming output always emphasizes the target sound. When it can be assumed that the sound source signals are uncorrelated with each other, the PSD of the l-th beamforming output signal is modeled by Equation (95).

Here, φ _{Sk, ω} represents the PSD of the kth sound source. D _{ζ (l), k, ω} represents an average of spatial sensitivity to the position of the kth sound source of the lth beamforming. The L _φ Yζ _{(l), ω} and the K phi _Sk, relationship _omega is modeled in equation (96).

In many cases, it is difficult to accurately estimate the number of sound sources in advance, so it is assumed that K ≒ L, and the beamforming signal group that has been picked up by emphasizing the place where noise is expected to arrive appropriately May be used.

  In order to estimate L local PSDs, the inverse problem of equation (96) is solved. When temporal sparsity is very high and it can be assumed that the sound source signals are uncorrelated with each other, it can be assumed that the relationship of Expression (96) holds for each time frame. By equation (97), the PSD of the sound source signal can be estimated for each frame.

By calculating ^ φ _{S, ω, τ} and ^ φ _{N, ω, τ} from the estimated local PSD ^ Φ _{S, ω, τ} , the Wiener filter can be calculated sequentially.

Here, alpha _{N, k, omega} is a coefficient for adjusting, empirically is often determined from the output value.

In the present invention, in order to output a signal in which the target sound source is clearly emphasized, the signal group after beam forming and the estimated local PSD ^ Φ _{S, ω, τ} are used as input feature amounts, and the prior SNR ^ ξ _{ω, τ} We introduce a statistical mapping model that outputs In recent years, a deep neural network is often used as one of statistical mapping models. Therefore, a deep neural network is used here.

  The points when the present invention is applied to sound source enhancement are (1) a configuration in which an estimated local PSD and a plurality of beamforming output power groups are used as inputs to a deep neural network, and a prior SNR is output; and (2) a deep neural network. The initial value of the network parameter is set in consideration of physical characteristics as in the conventional method.

[Fifth embodiment]
As illustrated in FIG. 13, the sound source emphasizing device of the fifth embodiment includes M microphones 10-1 to 10 -M, M frequency domain conversion units 11-1 to 11 -M, and L beam forming units. 12-1 to 12-L, a local PSD estimation unit 13, a DNN mapping unit 22, and a filtering unit 16. The sound source emphasizing apparatus according to the fifth embodiment is realized by the sound source emphasizing apparatus performing processes in steps described later.

  The sound source emphasis device is, for example, a special program configured by reading a special program into a known or dedicated computer having a central processing unit (CPU), a main storage device (RAM: Random Access Memory), and the like. Device. The sound source emphasizing apparatus executes each process under the control of the central processing unit, for example. The data input to the sound source emphasizing device and the data obtained in each process are stored in, for example, the main storage device, and the data stored in the main storage device is read as necessary and used for other processing. The In addition, at least a part of each processing unit of the sound source emphasizing device may be configured by hardware such as an integrated circuit.

  With reference to FIG. 14, the processing procedure of the sound source enhancement method of the fifth embodiment will be described.

In step S10, a microphone array including M microphones 10-1 to 10-M collects M observation signals x _m (n) (m = 1,..., M). Here, n represents the sample number of the discrete time signal. The observation signal x _m (n) is input to each of the frequency domain conversion units 11-1 to 11-M.

In step S11, the frequency domain transform unit 11-m (m = 1,..., M) sets each observation signal x _m (n) to a short time length (for example, about 256 samples when the sampling frequency is 16,000 Hz). And the discrete Fourier transform is performed in each frame, and the frequency domain observation signals X _{m, ω, τ} are output. Here, ω represents a frequency bin number, and τ represents a frame number. The output signal groups X1 _{, ω, τ} , X2 _{, ω, τ} ,..., XM _{, ω, τ} of the frequency domain transform units 11-1 to 11-M are supplied to the beam forming units 12-1 to 12-L. Each is entered.

In step S12, the beam forming units 12-1 to 12-L output signal groups X1 _{, ω, τ} , X2 _{, ω, τ} , ..., XM of the frequency domain transform units 11-1 to 11- _{M. , ω, τ} are processed to collect sounds by emphasizing sounds coming from angular regions in different directions, and the results are output. The beam forming unit 12-1 is used for direct sound enhancement, and outputs output signals Y _{ζ (1), ω, τ} by enhancing sound coming from a predetermined sound source direction. The remaining beam forming units 12-2 to 12-L are used for analysis of diffuse reverberation, and emphasize the sound coming from directions other than the sound source direction to output signal groups Y _{ζ (2), ω, τ} , …, Y _{ζ (L), ω, τ} are output. The output signal groups Y _{ζ (1), ω, τ} ,..., Y _{ζ (L), ω, τ} of the beam forming units 12-1 to 12-L are input to the local PSD estimation unit 13.

In step S13, the local PSD estimation unit 13 receives the output signal groups Y _{ζ (1), ω, τ} ,..., Y _{ζ (L), ω, τ} of the beam forming units 12-1 to 12-L as inputs. Then, local PSD ^ Φ _{S, ω, τ} is estimated according to the above equation (97). The estimated local PSD ^ Φ _{S, ω, τ} is input to the DNN mapping unit 22.

In step S22, the DNN mapping unit 22 receives the local PSD ^ Φ _{S, ω, τ} output from the local PSD estimation unit 13, and uses the network parameter z _ωi to estimate the prior SNR ^ ξ _{ω, τ} = ^ Find φ _{S, ω, τ} / ^ φ _{N, ω, τ} and output the result.

  Hereinafter, the process of the DNN mapping unit 22 will be described in detail. The DNN mapping unit 22 is composed of an N-layer deep neural network, as in the case of the direct ratio estimation. N may be about 4-5. First, feature values are set in the input layer of the deep neural network.

The dimension (number of nodes ⁾ of the vector q _ωi ⁽¹⁾ at this time is J ₁ = K × Q. If the network parameters z _ωi include Z _ωi ⁽²⁾ ,…, Z _ωi ^(N) , b _ωi ⁽²⁾ ,…, b _ωi ^(N) Is calculated.

Here, when the number of layers n-th layer is described as J _n,

It is. The activation function f ⁽ⁿ⁾ (•) is expressed as sigmoid function (when n = 2,…, N-1) and identity mapping function (when n = N) ).

The number of layers in the Nth layer is J _N = 1, and the estimated prior SNR is expressed by Equation (109).

Hereinafter, the prior SNR estimated using q _ωi ⁽¹⁾ as an input and the network parameter z _ωi is denoted as ζ (q _ωi ⁽¹⁾ ; z _ωi ). The network parameters are learned and designed in each frequency bin and each band.

Hereinafter, the optimization method of the deep neural network will be described. In the fifth embodiment, an initial value of the network parameter z _ωi is set at random, and the network parameter z _ωi is optimized so as to minimize the estimation error of the prior SNR based on back propagation. . A large number of observation signal sample data including the time frame direction is prepared, and teacher information including a local PSD for learning composed of a total of Θ data and a correct value of the prior SNR is described as follows.

  When applying each step of equations (101) and (102) to Θ sample data, it can be written in matrix form as follows:

Where b _ωi ⁽ⁿ⁾ 1 _Θ ^T represents the operation of arranging b _{ω i} ⁽ⁿ⁾ by Θ,

It is.

  As a measure for measuring an error between the output of the deep neural network and the prior SNR given as a correct answer, a square error function defined by Expression (115) is used.

Based on the back propagation error, the gradient of the network parameter is calculated sequentially from the output layer (n = N) to the input layer (n = 1). ^{_{- Ξ ωi = [- ξ ωi}} , 1, ..., - ξ ωi, Θ] When the respective sample data definitive delta delta _.omega.i in n-th layer _is obtained as follows ^(n).

Where f '(•) is the derivative of the function f (•),

Represents the product of each component of the matrix. The slope of the error function is calculated as follows:

  Finally, the parameters are updated based on the obtained gradient.

The update amounts ΔZ _ωi ⁽ⁿ⁾ and Δb _ωi ⁽ⁿ⁾ may be set as follows.

Where ΔZ _ωi ^{(n) +} , Δb _ωi ^{(n) +} is the previous update amount, ε is a learning coefficient, μ is a momentum coefficient to improve generalization performance and speed up learning, and λ is Weight decay. ε may be set to about 0.01, μ may be set to about 0.9, and λ may be set to about 0.0002.

In step S16, the filtering unit 16 receives the pre-SNR estimated value ^ ξ _{ω, τ} output from the DNN mapping unit 22, calculates a Wiener filter according to the above equation (94), and performs beamforming according to the above equation (93). By multiplying the output signal group Y _{ζ (1), ω, τ} ,..., Y _{ζ (L), ω, τ} by the Wiener filter, the output signals Z _{i, ω, τ} are output.

[Sixth embodiment]
In the fifth embodiment, an N-layer deep neural network that outputs local PSD ^ Φ _{S, ω, τ} as an input and outputs an estimated value of prior SNR ^ ξ _{ω, τ} is designed in each frequency bin and in each band. explained. In the sixth embodiment, a configuration using an N-layer deep neural network that receives a plurality of beamforming outputs as inputs and outputs an estimated value of a prior SNR will be described.

  As shown in FIG. 15, the sound source emphasizing apparatus according to the sixth embodiment includes M microphones 10-1 to 10-M, M frequency domain transform units 11-1 to 11-M, and L beam forming units. 12-1 to 12-L and the filtering unit 16 are provided in the same manner as in the fifth embodiment, and a DNN mapping unit 23 is further provided. The local PSD estimation unit 13 included in the sound source enhancement device of the fifth embodiment is not provided, and the outputs of the beamforming units 12-1 to 12 -L are input to the DNN mapping unit 23. The sound source emphasizing device according to the sixth embodiment is realized by the sound source emphasizing apparatus performing processing of each step described later.

In step S23, the DNN mapping unit 23 receives the output signal groups Y _{ζ (1), ω, τ} ,..., Y _{ζ (L), ω, τ} of the beam forming units 12-1 to 12-L as inputs, Using the network parameter z _ωi , an estimated value SNR of prior SNR ^ ξ _{ω, τ} is obtained and the result is output. The DNN mapping unit 23 is optimized in the same manner as in the fifth embodiment, using teacher information composed of Θ pieces of sample data and consisting of a beamforming output for learning and a correct value of a prior SNR. It is.

[Seventh embodiment]
In the fifth embodiment and sixth embodiment, setting the initial value of the network parameter z _.omega.i randomly. In the seventh embodiment, a method for setting the initial value of the network parameter z _ωi in consideration of physical characteristics as in the conventional method will be described. The prior SNR estimation technique in the conventional method is mainly composed of the following three steps.

(Step 1: Local PSD estimation processing) As shown in equation (97), the estimated value ^ Φ _{S, ω, τ} of the local PSD is _{calculated from} two or more beamforming output power groups φ _{Yζ (i), ωi, τ.} Ask.

(Step 2: Addition Processing) As shown in equations (98) and (99), PSDs φφ _{S, ω, τ} , ^ φ _{N, ω, τ} of the target sound and noise in the beamforming output are output.

(Step 3: Logarithmic domain ratio calculation process) As shown in equation (124), the estimated prior SNR value ^ ξ _{ω, τ} is output from ^ φ _{S, ω, τ} , ^ φ _{N, ω, τ} as follows: To do.

Since the above three-step processing can be regarded as physically corresponding to the processing of each layer, it is possible to determine the initial values of the network parameters with higher quality than those set at random. In the seventh embodiment, the sound source emphasizing device of the sixth embodiment is configured to set the initial value of the network parameter z _ωi . The optimization process is the same as in the sixth embodiment.

  As shown in FIG. 16, the sound source emphasizing apparatus according to the seventh embodiment includes M microphones 10-1 to 10-M, M frequency domain transform units 11-1 to 11-M, and L beam forming units. 12-1 to 12-L, the filtering unit 16, and the DNN mapping unit 23 are provided in the same manner as in the sixth embodiment, and further an initial value setting unit 33 is provided. The sound source emphasizing apparatus according to the seventh embodiment is realized by the sound source emphasizing apparatus performing processes in steps described later.

  The initial value setting unit 33 sets initial values of network parameters corresponding to each layer of the DNN mapping unit 23 as follows. As in the sixth embodiment, the value of the input layer is set as follows:

The processing of the second layer corresponds to (Step 1: Local PSD estimation processing). The number of layers in the second layer should be J ₂ ≧ L × Q. L is the number of beamforming, and Q is the number of frequency bins. The elements of D _ω ⁻¹ included in Expression (97) are defined as follows.

Here, ⁻¹ represents an inverse matrix when L = K, and a pseudo inverse matrix when L ≠ K.

  By assigning the corresponding coefficient to the network parameter, the initial value of the second layer can be set.

G _{ωi, 2} and B _{ωi, 2} are value range adjustment coefficients. G _{ωi, 2} is set so that the maximum value of Z _ωi ⁽²⁾ q _ωi ⁽¹⁾ is about 1-5. Further, when the output value of Z _ωi ⁽²⁾ q _ωi ⁽¹⁾ is 0 or less, B _{ωi, 2} is set between -5 and 0 in order to floor the value near 0. Thereafter, the output q _ωi ⁽²⁾ of the second layer is obtained by _performing the following calculation.

The processing of the third layer corresponds to (Step 2: Addition processing). The addition process can be expressed by writing the network parameters as follows. Note that the number of layers in the third layer is set to satisfy J ₃ ≧ 2.

G _{ωi, 3} and B _{ωi, 3} are value range adjustment coefficients. G _{ωi, 3} is set so that the maximum value of Z _ωi ⁽³⁾ q _ωi ⁽²⁾ is about 1-5. In _addition, B ωi, ₃ there is no problem at about 0. Thereafter, by calculating the equations (133) and (134), the output q _ωi ⁽³⁾ of the third layer is obtained.

The processing of the fourth layer corresponds to (Step 3: Logarithmic area ratio calculation processing). Logarithmic domain ratio calculation processing can be expressed by writing network parameters as follows. Note that the number of layers in the fourth layer (output layer) is J ₄ = 1.

Here, in order to correspond to the equation (124), Z _{ωi, 1,1} ⁽⁴⁾ is limited to a positive value, and Z _{ωi, 1,2} ⁽⁴⁾ is limited to a negative value. For example, the value is determined as follows.

At this time, the reference ^ φ _{S, ω, τ} and ^ φ _{N, ω, τ} may be calculated from one sample, or the average of the values calculated from many samples A value may be used. Further, the adjustment coefficient g _{ωi, 4} is obtained as follows.

  Finally, the output value is calculated as follows.

  In the network parameter initial value setting method described above, the number of layers is preferably set to be equal to or greater than the minimum unit number of signal processing operations + 1. In the above description, it is assumed that N = 4. However, if N is desired to be larger than 4, a redundant layer may be interposed. Here, the minimum unit number of signal processing operations is a signal such as addition or multiplication that is required when an equivalent signal processing operation (here, prior SNR estimation processing) is executed by a conventional deterministic method. This means the number of processing operations.

[Eighth embodiment]
In the seventh embodiment, the configuration in which the initial value of the network parameter z _ωi is set in the sound source enhancement device of the sixth embodiment has been described. In the eighth embodiment, the sound source emphasizing device of the fifth embodiment is configured to set an initial value of the network parameter z _ωi . The optimization process is the same as in the fifth embodiment.

  As illustrated in FIG. 17, the sound source emphasizing device according to the eighth embodiment includes M microphones 10-1 to 10 -M, M frequency domain conversion units 11-1 to 11 -M, and L beam forming units. 12-1 to 12-L, the local PSD estimation unit 13, the DNN mapping unit 22, and the filtering unit 16 are provided in the same manner as in the fifth embodiment, and further an initial value setting unit 32 is provided. The sound source emphasizing apparatus according to the eighth embodiment is realized by the processing of each step described later.

  In the seventh embodiment, it has been described that the process of the second layer corresponds to the local PSD estimation process. Therefore, in the eighth embodiment, it is sufficient to use the initial value settings for the third and subsequent layers. As in the fifth embodiment, the value of the input layer is set as follows:

  The processing after the expression (131) may be set as the initial values for the second and third layers. In the seventh embodiment, it has been described that it is better to set N ≧ 4. However, since the concept of setting the number of layers to be equal to or more than the minimum unit number +1 of the signal processing operation is the same, in the eighth embodiment, N It is desirable to set ≧ 3.

  As in the seventh and eighth embodiments, by appropriately setting the initial values of the network parameters, it is possible to optimize the parameters while having the physical meaning of each layer. As a result, an effect of reducing the possibility of outputting an outlier even if the learning data is small to some extent is expected. That is, there is an effect that it is possible to design a deep neural network that is less dependent on the environment even if there is little learning data.

[Ninth embodiment]
The ninth embodiment is a sound source information as a superordinate concept that includes the direct ratio estimation technology described in the first embodiment to the fourth embodiment and the sound source enhancement technology described in the fifth embodiment to the eighth embodiment. The estimation technique will be described.

  The sound source information estimation device of the ninth embodiment includes, for example, a sound source feature extraction unit and a sound source information estimation unit. The sound source information estimation apparatus of the ninth embodiment is realized by this sound source information estimation apparatus performing the processing of each step described later.

  The sound source feature extraction unit extracts sound source features of each frequency domain observation signal from a plurality of frequency domain observation signals picked up by emphasizing sounds coming from a plurality of angular regions in different directions. The sound source feature extraction unit corresponds to the beam forming units 12-1 to 12-2 and the local PSD estimation unit 13 in the direct ratio estimation apparatuses of the first embodiment and the fourth embodiment. The direct ratio estimation apparatus according to the embodiment corresponds to the beam forming units 12-1 to 12-2. In the sound source emphasizing devices of the fifth embodiment and the eighth embodiment, they correspond to the beam forming units 12-1 to 12-L and the local PSD estimating unit 13, and the sound source emphasizing devices of the sixth embodiment and the seventh embodiment. Then, it corresponds to the beam forming units 12-1 to 12-L.

  The sound source information estimation unit obtains an estimated value of the sound source information by inputting the sound source characteristics of each frequency domain observation signal to the statistical mapping model. At this time, the statistical mapping model emphasizes sound coming from a plurality of angular regions in different directions and extracts sound source characteristics extracted from a plurality of frequency domain acoustic signals collected and collected from each frequency domain acoustic signal. The parameters are learned using correct values. The sound source information estimation unit corresponds to the DNN mapping unit 20 in the direct ratio estimation apparatus of the first embodiment and the fourth embodiment, and the DNN mapping in the direct ratio estimation apparatus of the second embodiment and the third embodiment. It corresponds to the part 21. Further, the sound source emphasizing devices of the fifth and eighth embodiments correspond to the DNN mapping unit 22, and the sound source emphasizing devices of the sixth embodiment and the seventh embodiment correspond to the DNN mapping unit 23.

  In the above embodiment, the example in which the statistical mapping model is configured by the deep neural network has been described. However, the direct ratio estimation technique of the first embodiment and the second embodiment, and the fifth embodiment and the sixth embodiment. The statistical mapping model in the sound source enhancement technique and the sound source information estimation technique in the ninth embodiment is not limited to the deep neural network, and other statistical mapping models can be used. Examples of other statistical mapping models include a mixed normal distribution (Gaussian Mixture Model: GMM). In the direct ratio estimation technique of the third embodiment and the fourth embodiment and the sound source enhancement technique of the seventh embodiment and the eighth embodiment, a technique for setting initial values of network parameters of the deep neural network is used. As such, the statistical mapping model is limited to deep neural networks.

[Tenth embodiment]
In the above embodiment, the hardware structure of the microphone array has not been particularly limited. An object of the present embodiment is to increase the robustness of the learned deep neural network network parameters and to improve the estimation performance of sound source information by limiting the hardware structure of the microphone array to be symmetric. The processing procedure is the same as that of each embodiment except that the hardware configuration is limited. Therefore, in the following, a specific hardware configuration example of a symmetric microphone array and why this configuration is a deep neural network. Explains whether the network parameters are more robust.

  FIG. 18 shows an example of an array structure having symmetry. Here, the symmetry refers to point symmetry in a two-dimensional or three-dimensional space. For example, when M microphones are arranged in a straight line, they are one-dimensional and cannot have symmetry. In the case of a two-dimensional structure, the case where microphones are arranged at equal intervals on the circumference (that is, the vertex position of a regular polygon) is applicable. Further, in the case of a three-dimensional structure, for example, a case where a microphone is present at the apex position of a regular polyhedron is applicable. In FIG. 18, regular triangles, squares, regular hexagons, and octagons are shown as examples of the two-dimensional structure, and regular tetrahedrons, regular hexahedrons, regular octahedrons, regular dodecahedrons, and regular icosahedrons are exemplified as three-dimensional structures. Although cases are shown, the present invention is not limited to these structures. When the microphone itself has directivity, the orientation of the element is limited so as to maintain symmetry.

The effect of giving symmetry to the structure of the microphone array will be described by taking the case of speech enhancement as an example. As explained in «Sound source enhancement method 1: Beam forming», the basic method for enhancing the target sound is to perform beam forming as shown in equation (84), and then to a Wiener like equation (93). Applying filtering. At that time, the PSD of the target sound and noise, or the prior SNRξω _{, τ,} which is the ratio thereof, is required, and these can be obtained by the calculation as shown in Equation (97). In each embodiment, a deep neural network has been used, which basically corresponds to the automatic estimation of this flow. By having symmetry structure of the microphone array, the sensitivity matrix D _omega the same regardless of the direction of arrival of the target sound. As a result, the processing flow for emphasizing the target sound is determined independently of the arrival direction of the target sound. Therefore, even when learning and estimating a flow that emphasizes the target sound using a deep neural network, the network parameters are determined independently of the direction of arrival of the target sound. Network parameter learning progresses without preparing a large amount. However, it is assumed that the direction of arrival of the target sound is known when performing beamforming. In this way, the robustness of the deep neural network can be enhanced by using the symmetrical microphone array, and the estimation performance of the sound source information can be further enhanced.

  The present invention is not limited to the above-described embodiment, and it goes without saying that modifications can be made as appropriate without departing from the spirit of the present invention. The various processes described in the above embodiment may be executed not only in time series according to the order of description, but also in parallel or individually as required by the processing capability of the apparatus that executes the processes or as necessary.

[Program, recording medium]
When various processing functions in each device described in the above embodiment are realized by a computer, the processing contents of the functions that each device should have are described by a program. Then, by executing this program on a computer, various processing functions in each of the above devices are realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used.

  This program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its storage device. When executing the process, the computer reads a program stored in its own recording medium and executes a process according to the read program. As another execution form of the program, the computer may directly read the program from a portable recording medium and execute processing according to the program, and the program is transferred from the server computer to the computer. Each time, the processing according to the received program may be executed sequentially. A configuration in which the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes a processing function only by an execution instruction and result acquisition without transferring a program from the server computer to the computer. It is good. Note that the program in this embodiment includes information that is used for processing by an electronic computer and that conforms to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer).

  In this embodiment, the present apparatus is configured by executing a predetermined program on a computer. However, at least a part of these processing contents may be realized by hardware.

10-1 to 10-M Microphones 11-1 to 11-M Frequency domain conversion units 12-1 to 12-L Beam forming unit 13 Local PSD estimation unit 14 Power ratio estimation unit 15 Prior SNR calculation unit 16 Filtering units 20 to 23 DNN mapping unit 30 to 33 Initial value setting unit

Claims (16)

A sound source feature extraction unit that receives a plurality of frequency domain observation signals picked up by emphasizing sounds coming from a plurality of angular regions in different directions, and extracts a power spectrum density of each frequency domain observation signal;
A sound source information estimation unit that inputs the power spectral density of each frequency domain observation signal to a statistical mapping model to obtain an estimate of the direct ratio,
Including
In the statistical mapping model, N is a unit number of signal processing operations constituting a predetermined direct ratio estimation processing operation, and a plurality of frequency regions in which sound is collected by enhancing sounds coming from a plurality of angular regions in different directions It is a deep neural network of N + 1 layers or more that learned parameters using the power spectral density extracted from the acoustic signal and the correct value of the direct ratio obtained from each frequency domain acoustic signal,
The input layer of the above deep neural network is set to the power spectral density of each frequency domain observation signal,
The intermediate layer of the deep neural network is learned to be composed of a first layer corresponding to local PSD estimation processing, a second layer corresponding to frequency addition processing, and a third layer corresponding to logarithmic domain ratio calculation processing. As shown, the initial values of the network parameters for each layer are set taking physical characteristics into account,
Sound source information estimation device. A sound source feature extraction unit that generates a plurality of frequency domain observation signals by emphasizing sounds coming from a plurality of angular regions in different directions with respect to the input observation signal, and extracts the power of each frequency domain observation signal;
A local PSD estimation unit that estimates the power spectral density of each frequency domain observation signal using the power of each frequency domain observation signal;
A sound source information estimation unit that inputs the power spectral density of each frequency domain observation signal to a statistical mapping model to obtain an estimate of the direct ratio,
Including
In the statistical mapping model, N is a unit number of signal processing operations constituting a predetermined direct ratio estimation processing operation, and a plurality of frequency regions in which sound is collected by enhancing sounds coming from a plurality of angular regions in different directions It is a deep neural network of N + 1 layers or more that learned parameters using the power spectral density extracted from the acoustic signal and the correct value of the direct ratio obtained from each frequency domain acoustic signal,
The input layer of the above deep neural network is set to the power spectral density of each frequency domain observation signal,
The intermediate layer of the deep neural network has an initial value of the network parameter of each layer so that it is learned to be composed of a first layer corresponding to frequency addition processing and a second layer corresponding to logarithmic domain ratio calculation processing. , Set with physical properties in mind,
Sound source information estimation device. A sound source feature extraction unit that receives a plurality of frequency domain observation signals picked up by emphasizing sounds coming from a plurality of angular regions in different directions, and extracts a power spectrum density of each frequency domain observation signal;
A sound source information estimator that inputs the power spectral density of each frequency domain observation signal to a statistical mapping model and obtains an estimate of the SN ratio;
A filtering unit that calculates a gain coefficient for each frequency band from the estimated value of the SN ratio and multiplies the power spectrum density of each corresponding frequency band of the frequency domain observation signal;
Including
In the statistical mapping model, N is the number of signal processing operations constituting a predetermined signal-to-noise ratio estimation processing operation, and a plurality of frequency domain sounds collected by emphasizing sounds coming from angle regions in different directions It is a deep neural network of N + 1 layers or more that learned parameters using the power spectral density extracted from the signal and the correct value of the SN ratio obtained from each frequency domain acoustic signal,
The input layer of the above deep neural network is set to the power spectral density of each frequency domain observation signal,
The intermediate layer of the deep neural network is learned to include a first layer corresponding to local PSD estimation processing, a second layer corresponding to addition processing, and a third layer corresponding to logarithmic domain ratio calculation processing. As shown, the initial values of the network parameters of each layer are set taking physical characteristics into account,
Sound source information estimation device. A sound source feature extraction unit that generates a plurality of frequency domain observation signals by emphasizing sounds coming from a plurality of angular regions in different directions with respect to the input observation signal, and extracts the power of each frequency domain observation signal;
A local PSD estimation unit that estimates the power spectral density of each frequency domain observation signal using the power of each frequency domain observation signal;
A sound source information estimator that inputs the power spectral density of each frequency domain observation signal to a statistical mapping model and obtains an estimate of the SN ratio;
A filtering unit that calculates a gain coefficient for each frequency band from the estimated value of the SN ratio and multiplies the power spectrum density of each corresponding frequency band of the frequency domain observation signal;
Including
In the statistical mapping model, N is the number of signal processing operations constituting a predetermined signal-to-noise ratio estimation processing operation, and a plurality of frequency domain sounds collected by emphasizing sounds coming from angle regions in different directions It is a deep neural network of N + 1 layers or more that learned parameters using the power spectral density extracted from the signal and the correct value of the SN ratio obtained from each frequency domain acoustic signal,
The input layer of the above deep neural network is set to the power spectral density of each frequency domain observation signal,
The intermediate layer of the deep neural network is learned to be composed of a first layer corresponding to the addition process and a second layer corresponding to the logarithmic domain ratio calculation process, so that the initial values of the network parameters of each layer are Set with physical characteristics in mind,
Sound source information estimation device. The sound source information estimation device according to claim 1,
Q is the number of frequency bins, ω is the frequency bin number, P _{BF, l, ω} is the power spectral density of the l-th beamforming output, and G _{l, Ω, ω} is the l-th beamforming filter for the direction Ω. Sensitivity, P _{D, ω} is the power spectral density of the direct sound, P _{R, ω} is the power spectral density of the reverberation, f ⁽ⁿ⁾ is the activation function of the nth layer, and J _n is the layer of the nth layer , G ₂ , g ₃ , B ₂ , B ₃ are predetermined value adjustment factors, ^T represents transposition,
The input layer of the above deep neural network is set to the following formula,

The intermediate layer of the deep neural network is
The initial values of the first layer network parameters are set as

The output of the first layer is calculated by

The initial values of the second layer network parameters are set to

The output of the second layer is calculated by

The initial value of the network parameter of the third layer is set as follows:

Sound source information estimation device. The sound source information estimation device according to claim 2,
Q is the number of frequency bins, ω is the frequency bin number, P _{D, ω} is the power spectral density of the direct sound, P _{R, ω} is the power spectral density of the reverberation, and f ⁽ⁿ⁾ is the activation of the nth layer J _n is the number of layers in the n-th layer, g ₃ and B ₃ are predetermined width adjustment factors, ^T represents transposition, ^ represents an estimated value,
The input layer of the above deep neural network is set to the following formula,

The intermediate layer of the deep neural network is
The initial values of the first layer network parameters are set as

The output of the first layer is calculated by

The initial values of the network parameters of the second layer are set as follows:

Sound source information estimation device. The sound source information estimation device according to claim 3,
L is the number of beamforming, K is the number of sound sources, Q is the number of frequency bins, ω is the frequency bin number, τ is the frame number, and φ _{Yζ (l), ω, τ} is the output of the l-th beamforming D _{ζ (l), k, ω} is the average of spatial sensitivity to the position of the kth sound source of the lth beamforming, ψk _{, l, ω} is a value defined by

_Let ^ φ _{S, ω, τ be} an estimate of the power spectrum density of the target sound, ^ φ _{N, ω, τ be} an estimate of the noise power spectral density, α _{N, k, ω} be a predetermined coefficient, f ⁽ⁿ⁾ is the activation function of the n-th layer, the J _n is the number of layers n-th layer, · ^T denotes _{transposition, g ωi, 2, g ωi} , 3, B ωi, 2, B ωi, 3 Is a predetermined price range adjustment factor,
The input layer of the above deep neural network is set to the following formula,

The intermediate layer of the deep neural network is
The initial values of the first layer network parameters are set as

The output of the first layer is calculated by

The initial values of the second layer network parameters are set to

The output of the second layer is calculated by

The initial value of the network parameter of the third layer is set as follows:

Sound source information estimation device. The sound source information estimation device according to claim 4,
_Let K be the number of sound sources, Q be the number of frequency bins, ω be the frequency bin number, τ be the frame number, ^ φ _{S, ω, τ be} the estimated power spectral density of the target sound, ^ φ _{N, ω , τ} is an estimate of the noise power spectral density, α _{N, k, ω} is a predetermined coefficient, f ⁽ⁿ⁾ is the activation function of the nth layer, J _n is the number of layers of the nth layer, ^T represents transposition, and g _{ωi, 3} , B _{ωi, 3} is a predetermined value adjustment factor,
The input layer of the above deep neural network is set to the following formula,

The intermediate layer of the deep neural network is
The initial values of the first layer network parameters are set as

The output of the first layer is calculated by

The initial values of the network parameters of the second layer are set as follows:

Sound source information estimation device. The sound source information estimation device according to any one of claims 1, 3, 5, and 7,
In the deep neural network, an input layer value is input to the first layer, an output of the first layer is input to the second layer, and an output of the second layer is input to the third layer. And the output is calculated based on the output of the third layer,
Sound source information estimation device. The sound source information estimation device according to any one of claims 2, 4, 6, and 8,
In the deep neural network, the value of the input layer is input to the first layer, the output of the first layer is input to the second layer, and the output is calculated based on the output of the second layer. Configured to be
Sound source information estimation device. The sound source information estimation device according to any one of claims 1 to 10,
The plurality of frequency domain acoustic signals are collected using a microphone array in which each microphone is arranged at the apex position of a regular polygon or regular polyhedron.
Sound source information estimation device. A sound source feature extraction unit that extracts a power spectrum density of each frequency domain observation signal by inputting a plurality of frequency domain observation signals picked up by emphasizing sounds coming from a plurality of angular regions in different directions. When,
A sound source information estimation unit that inputs the power spectral density of each frequency domain observation signal to a statistical mapping model to obtain an estimate of the direct ratio;
Including
In the statistical mapping model, N is a unit number of signal processing operations constituting a predetermined direct ratio estimation processing operation, and a plurality of frequency regions in which sound is collected by enhancing sounds coming from a plurality of angular regions in different directions It is a deep neural network of N + 1 layers or more that learned parameters using the power spectral density extracted from the acoustic signal and the correct value of the direct ratio obtained from each frequency domain acoustic signal,
The input layer of the above deep neural network is set to the power spectral density of each frequency domain observation signal,
The intermediate layer of the deep neural network is learned to be composed of a first layer corresponding to local PSD estimation processing, a second layer corresponding to frequency addition processing, and a third layer corresponding to logarithmic domain ratio calculation processing. As shown, the initial values of the network parameters for each layer are set taking physical characteristics into account,
Sound source information estimation method. A sound source whose sound source feature extraction unit emphasizes sounds coming from multiple angular regions in different directions with respect to the input observation signal, generates a plurality of frequency domain observation signals, and extracts the power of each frequency domain observation signal A feature extraction step;
A local PSD estimation step in which a local PSD estimation unit estimates the power spectral density of each frequency domain observation signal using the power of each frequency domain observation signal;
A sound source information estimation unit that inputs the power spectral density of each frequency domain observation signal to a statistical mapping model to obtain an estimate of the direct ratio;
Including
In the statistical mapping model, N is a unit number of signal processing operations constituting a predetermined direct ratio estimation processing operation, and a plurality of frequency regions in which sound is collected by enhancing sounds coming from a plurality of angular regions in different directions It is a deep neural network of N + 1 layers or more that learned parameters using the power spectral density extracted from the acoustic signal and the correct value of the direct ratio obtained from each frequency domain acoustic signal,
The input layer of the above deep neural network is set to the power spectral density of each frequency domain observation signal,
The intermediate layer of the deep neural network has an initial value of the network parameter of each layer so that it is learned to be composed of a first layer corresponding to frequency addition processing and a second layer corresponding to logarithmic domain ratio calculation processing. , Set with physical properties in mind,
Sound source information estimation method. A sound source feature extraction unit that extracts a power spectrum density of each frequency domain observation signal by inputting a plurality of frequency domain observation signals picked up by emphasizing sounds coming from a plurality of angular regions in different directions. When,
A sound source information estimation unit inputs a power spectral density of each frequency domain observation signal to a statistical mapping model to obtain an SN ratio estimate,
A filtering step in which a filtering unit calculates a gain coefficient for each frequency band from the estimated value of the SN ratio and multiplies the power spectral density of each corresponding frequency band of the frequency domain observation signal;
Including
In the statistical mapping model, N is the number of signal processing operations constituting a predetermined signal-to-noise ratio estimation processing operation, and a plurality of frequency domain sounds collected by emphasizing sounds coming from angle regions in different directions It is a deep neural network of N + 1 layers or more that learned parameters using the power spectral density extracted from the signal and the correct value of the SN ratio obtained from each frequency domain acoustic signal,
The input layer of the above deep neural network is set to the power spectral density of each frequency domain observation signal,
The intermediate layer of the deep neural network is learned to include a first layer corresponding to local PSD estimation processing, a second layer corresponding to addition processing, and a third layer corresponding to logarithmic domain ratio calculation processing. As shown, the initial values of the network parameters for each layer are set taking physical characteristics into account,
Sound source information estimation method. A sound source whose sound source feature extraction unit emphasizes sounds coming from multiple angular regions in different directions with respect to the input observation signal, generates a plurality of frequency domain observation signals, and extracts the power of each frequency domain observation signal A feature extraction step;
A local PSD estimation step in which a local PSD estimation unit estimates the power spectral density of each frequency domain observation signal using the power of each frequency domain observation signal;
A sound source information estimation unit inputs a power spectral density of each frequency domain observation signal to a statistical mapping model to obtain an SN ratio estimate,
A filtering step in which a filtering unit calculates a gain coefficient for each frequency band from the estimated value of the SN ratio and multiplies the power spectral density of each corresponding frequency band of the frequency domain observation signal;
Including
In the statistical mapping model, N is the number of signal processing operations constituting a predetermined signal-to-noise ratio estimation processing operation, and a plurality of frequency domain sounds collected by emphasizing sounds coming from angle regions in different directions It is a deep neural network of N + 1 layers or more that learned parameters using the power spectral density extracted from the signal and the correct value of the SN ratio obtained from each frequency domain acoustic signal,
The input layer of the above deep neural network is set to the power spectral density of each frequency domain observation signal,
The intermediate layer of the deep neural network is learned to be composed of a first layer corresponding to the addition process and a second layer corresponding to the logarithmic domain ratio calculation process, so that the initial values of the network parameters of each layer are Set with physical characteristics in mind,
Sound source information estimation method.   The program for functioning a computer as a sound source information estimation apparatus in any one of Claim 1 to 11.