JP6182169B2 - Sound collecting apparatus, method and program thereof - Google Patents

Description

  The present invention relates to a microphone array signal processing technique. More specifically, the present invention relates to a microphone array sound collection technique in which sounds generated in a plurality of areas are separated for each area and simultaneously enhanced.

  For example, as shown in FIG. 1, 360 ° is divided into L predetermined areas (where L is an integer of 2 or more), and the sound generated in each area is recorded separately. If it is possible, it will be easy to create minutes for each speaker or for each seat in a meeting. In addition, video such as interactive panoramic video technology (see, for example, Patent Document 1) that reproduces video in a desired direction of a viewer by selectively combining video recorded using an apparatus such as an omnidirectional camera. When combined with a viewing system, it is also possible to reproduce only the sound existing in the viewing direction. In this specification, “speech” is not limited to a voice uttered by a person, but refers to a general “sound” such as a musical sound or an environmental noise as well as a voice of a person or an animal.

  Such a technology for enhancing and collecting sound coming from a specific direction / region is called “narrow-directed speech enhancement technology” and has been researched and developed conventionally. Typical narrow-directional speech enhancement uses multiple microphones and signal processing, such as those that use the phase difference of the sound pressure applied to the front and back of the vibration plate, such as directional microphones, and beamformers that use linear filtering using a microphone array (Hereinafter referred to as “array sound collection”).

  The relationship between the direction of the microphone and the sensitivity of the microphone is called directivity. The sharper the directivity in a certain direction (target direction), the more the sound in a narrow area including that direction is emphasized, and the other areas The generated voice can be suppressed. FIG. 3 shows a narrow-directional speech enhancement using a linear filter designed by the maximum likelihood method described in Non-Patent Document 1, in which six microphones are arranged at the apex of a regular hexagon on the same plane at intervals of 4.0 cm. This is an example of directivity. It can be seen that the width from the target direction where the gain is maximized to the direction where the gain is reduced to −15 dB is about ± 75 °. Hereinafter, in this specification, for the sake of convenience, an area from when the directivity decreases to a predetermined value (for example, −15 dB) from the gain in the target direction is referred to as a “beam area”.

  Accordingly, when a plurality of narrow-directional speech enhancements are performed at a narrow interval (a state in which an angle between adjacent target directions is small), an overlap region (hereinafter also referred to as “superimposition region”) is generated in the beam region as shown in FIG. When a sound source exists in the overlap region, the same sound source is included in both beam outputs. FIG. 4 is an example of an output waveform in which a sound source having an arrival direction of 30 ° is collected by two unidirectional microphones having a first output target direction of 0 ° and a second output target direction of 60 °. . From the output waveform, it can be seen that the sound source is included in each beam at substantially the same volume.

  This indicates that even if narrow direction speech enhancement of a plurality of areas is executed independently, recording with divided areas cannot be performed as shown in FIG.

JP 2012-209634 A

Tadashi Asano, “Sound Array Signal Processing-Localization, Tracking and Separation of Sound Sources”, Corona, 2011, pp.82-83.

  In order to solve the above-described problem, it is necessary for each narrow-directional speech enhancement signal processor to cooperate to separate and enhance speech for each region. In other words, there is a need for a signal processing technique for controlling the speech in the overlap region, such as suppressing the speech enhanced by a certain beam as noise in other beams. In view of the situation as described above, the present invention provides an array sound collection technique by nonlinear filtering that suppresses the sound source output of one beam when two beam outputs include the input from the same sound source.

  The problem to be solved by the present invention occurs when a sound source is present in the overlap region as in the sound source 1 of FIG. In this case, it is necessary to suppress the sensitivity to the sound source 1 in either the beam region 1 or the beam region 2.

  For this purpose, it is necessary to determine whether or not a sound source exists in the overlap region. As the easiest method, it is conceivable to compare the outputs of the beam areas 1 and 2 and suppress one of them if the similarity is a certain level or higher. However, if a sound source that affects only one beam region, such as the sound source 2, exists simultaneously with the sound source 1, the similarity is low, and the presence or absence of a sound source in the overlap region cannot be determined by simple comparison.

  The present invention discriminates whether or not a sound source exists in the overlap region without using the similarity, and when a sound source exists in the overlap region, the signal obtained from any of the beam regions forming the overlap region is used. On the other hand, an object of the present invention is to provide a sound collection device, a method and a program for performing processing for suppressing sensitivity to a sound source existing in an overlap region.

  In order to solve the above-described problems, according to one aspect of the present invention, a sound collection device is configured such that M and L are each an integer equal to or greater than 2, and L microphones are configured with M microphones. L main beam regions each having a direction as a target direction are formed, and sound is collected for each main beam region. When the sound collection device detects that a sound source exists in a sub beam region whose target direction is a superposition region of two adjacent main beam regions in a state where L main beam regions are formed, the sound source By determining based on the feature amount of the enhancement signal that emphasizes the sound in the main beam region and the sub beam region, which of the two main beam regions is closer to the main beam region, it is determined that the main beam region is not close to the sound source. Adjustment is performed so as to suppress the sound of the sound source included in the signal on the beam region side.

  In order to solve the above-described problem, according to another aspect of the present invention, a sound collection method is performed by using a microphone array configured with M microphones, where M and L are each an integer of 2 or more. L main beam areas are formed, each of which is a target direction, and sound is collected for each main beam area. When the sound collection method detects that a sound source exists in a sub-beam region whose target direction is an overlap region of two adjacent main beam regions in a state where L main beam regions are formed, the sound source By determining based on the feature amount of the enhancement signal that emphasizes the sound in the main beam region and the sub beam region, which of the two main beam regions is closer to the main beam region, it is determined that the main beam region is not close to the sound source. Adjustment is performed so as to suppress the sound of the sound source included in the signal on the beam region side.

  According to the present invention, it is determined whether or not a sound source exists in the overlap region without using the similarity, and if a sound source exists in the overlap region, it is obtained from any of the beam regions that form the overlap region. There is an effect that it is possible to perform processing for suppressing the sensitivity to the sound source existing in the overlap region on the signal.

The figure which shows the example of a division | segmentation of a sound collection area. The figure which shows the example of a division | segmentation of the sound collection area by the independent narrow directivity audio enhancement technique. The figure which shows the example of the directivity of narrow directivity voice emphasis. The figure which shows the output example of the narrow directivity audio | voice emphasis when a sound source exists in an overlap area | region. The figure which shows the example of arrangement | positioning when a sound source exists in an overlap area | region. The figure which shows the detection of the sound source of the overlap area | region by a sub beam. The functional block diagram of the sound collection device which concerns on 1st embodiment. The figure which shows the example of the processing flow of the sound collection device which concerns on 1st embodiment. The figure which shows the structural example of a beam formation part. The figure which shows the example of the processing flow of a winner gain calculation part. The figure which shows the structural example of a winner gain calculation part. The figure which shows the structural example of a beam reforming part. The figure which shows the example of the processing flow of the beam reforming part of 1st embodiment.

Hereinafter, embodiments of the present invention will be described. In the drawings used for the following description, constituent parts having the same function and steps for performing the same process are denoted by the same reference numerals, and redundant description is omitted. In the following explanation, the symbols `` <sup>-</sup> '', `` ~ '', etc. used in the text should be described immediately above the character immediately before, but are described immediately after the character due to restrictions on the text notation. . In the formula, these symbols are written in their original positions. Further, the processing performed for each element of a vector or matrix is applied to all elements of the vector or matrix unless otherwise specified.

<First embodiment>
In this embodiment, as shown in FIG. 6, a beam having an overlap region in a target direction (for convenience, hereinafter referred to as a sub beam) is used in addition to an output beam (for convenience, hereinafter referred to as a main beam). Each beam may be formed by beam forming using a microphone array, or may be formed using a single directional microphone. When a sound source is detected in the sub-beam region, a nonlinear filter that suppresses the detected sound source from one output of the adjacent main beam region is generated and applied. In the present embodiment, a time frequency mask using a Wiener filter is used as the nonlinear filter.

  Prior to the description of the present embodiment, narrow-directional speech enhancement by linear filtering using a microphone array and narrow-directional speech enhancement by a unidirectional microphone, which are prerequisite technologies, will be described and formulated.

  Narrow-directed speech enhancement by linear filtering using a microphone array is performed by superimposing the signal collected by each microphone with a filter containing information on time difference or sound pressure level difference, and then superimposing the sound in the target direction. To emphasize. Unlike acoustic tube microphones and parabola microphones, array sound collection performs sound enhancement by signal processing, so that sound in an arbitrary direction can be enhanced without mechanically rotating the microphone.

Now, consider that P number of source signal s _p (t) is picked up by using the M microphones, formed at the same time L direction of the beam space. Here, P is any integer greater than 1, M and L are each any integer greater than 2, and p = 1, 2,..., P, t are indices representing discrete time.

ただし、m=1,2,…,Mであり、*は畳み込み演算である。したがって、時間領域の収音信号x_m(t)を離散フーリエ変換して得られる周波数領域の収音信号X_m(ω,k)は以下のように記述できる。

ここで、A_p,m(ω)とS_p(ω,k)はそれぞれa_p,mとs_p(t)をフーリエ変換して得られる周波数領域の伝達特性と音源、kは時間フレームのインデックス、ωは周波数ビンのインデックスを表す。さらに上付き文字のTを転置とし、x^-(ω,k)=(X₁(ω,k),…,X_M(ω,k))^T、s^-(ω,k)=(S₁(ω,k),…,S_P(ω,k))^T、a^- _p(ω)=(A_p,1(ω),…,A_p,M(ω))^T、A^-(ω)=(a^- ₁(ω),…,a^- _P(ω))のように行列形式で書き直すと、x^-(ω,k)は以下のように記述できる。
x^-(ω,k)=A^-(ω)s^-(ω,k) (3) The collected sound signal x _m (t) picked up by M microphones can be described as follows using transfer characteristics a _{p, m} from the p-th sound source position to the m-th microphone.

However, m = 1, 2,..., M, and * is a convolution operation. Therefore, the frequency domain sound pickup signal X _m (ω, k) obtained by discrete Fourier transform of the time domain sound pickup signal x _m (t) can be described as follows.

_Here, A _{p, m} (ω) and S _p (ω, k), respectively a _p, transfer characteristics and a sound source in the frequency domain obtained by _m and s _p (t) is the Fourier transform, k is the time frame The index, ω, represents the frequency bin index. In addition to the T superscript and ^{transpose, x - (ω, k)} = (X 1 (ω, k), ..., X M (ω, k)) T, s - (ω, k) = (S 1 (ω, k), ..., S P (ω, k)) T, a - p (ω) = (A p, 1 (ω), ..., A p, M (ω)) T, A - (ω _{^{) = (a - 1 (ω}} ), ..., a - rewritten in matrix form as ^{_{P (ω)), x -}} (ω, k) can be described as follows.
^{x - (ω, k) =} A - (ω) s - (ω, k) (3)

と書くことができ、またw^-(ω,θ)=(W₁(ω,θ),…,W_M(ω,θ))^Hと置くことで行列形式を用いて
Y(ω,k,θ)=w^-(ω,θ)^Hx^-(ω,k) (5)
と記述できる。ここで上付き文字のHはエルミート転置を表す。さらに、L方向の目的方向{θ_l|l∈1,…,L}の出力信号Y(ω,k,l)も、y^-(ω,k)=(Y(ω,k,1),…,Y(ω,k,L))^T、W^-(ω)=(w^-(ω,θ₁),…,w(ω,θ_L))^Hと置くことで
y^-(ω,k)=W^-(ω)x^-(ω,k) (6)
と書くことができる。 The array collected signal enhancement technique by linear filtering enhances the sound in the target direction θ by convolving a linear filter w _m (θ) with the collected signal x _m (t). This linear filter can be designed by existing techniques such as a method of compensating for the positional difference of the microphone and a method of minimizing the sound power other than the target direction. When a frequency domain linear filter obtained by performing discrete Fourier transform on the time domain linear filter w _m (θ) is W _m (ω, θ), a narrow directivity enhancement signal (hereinafter also referred to as “emphasis signal”) Y ( (ω, k) is

And it can be written, also ^{w - (ω, θ) =} (W 1 (ω, θ), ..., W M (ω, θ)) using a matrix form by placing the ^H
Y (ω, k, θ) = w - (ω, θ) H x - (ω, k) (5)
Can be described. Here, the superscript H represents Hermitian transpose. Further, the output signal Y (ω, k, l) of the target direction {θ _l | l∈1,..., L} in the L direction is also y ⁻ (ω, k) = (Y (ω, k, 1), ..., Y (ω, k, L)) T, W - (ω) = (w - (ω, θ 1), ..., w (ω, θ L) by placing a) ^{H
y - (ω, k) =} W - (ω) x - (ω, k) (6)
Can be written.

と近似的に記述できることが知られており、α=1で単一指向性マイクロホンの指向特性となる。ここでθは、基準となる方向に対するマイクロホンと音源とのなす角である。
（参考文献１）中島 平太郎編, ”応用電気音響-”, コロナ社, 1979年、pp.16-17. The directional microphone realizes directivity by using, for example, a phase difference between sound pressures applied to the front and back of the vibration plate. According to Reference 1, the directivity is

It is known that the directional characteristics of a unidirectional microphone can be obtained when α = 1. Here, θ is an angle formed by the microphone and the sound source with respect to the reference direction.
(Reference 1) Hirataro Nakajima, “Applied electroacoustics”, Corona, 1979, pp.16-17.

ただし、θ_pは、基準となる方向に対するマイクロホンアレイの中心と音源pとのなす角である。さらに、h^- _p=(H₁(θ_p),…,H_M(θ_p))^T（ただし、H_m(θ_p)は、角θ_pにおけるマイクロホンmの音源pに対する指向性を表す）、θ^-=(θ₁,…,θ_P)、H^-(θ^-)=(h^- ₁,…,h^- _P)、s^-(ω,k,θ^-)=(S₁(ω,k,θ₁),…,S_P(ω,k,θ_P))^Tと置くと、
x^-(ω,k)=prod[H^-(θ^-),A^-(ω)]s^-(ω,k,θ^-) (9)
と書くことができる。
ここでprod[A,B]は2つの行列A,Bの要素ごとの積を表す。 Assuming that the directional microphone does not change the phase of the sound source and that the directivity is constant at all frequencies, the collected sound signal X _m (ω, k) and the sound source signal S _p (ω, k, θ _p ) The relationship can be described as follows:

Here, θ _p is an angle formed by the center of the microphone array and the sound source p with respect to the reference direction. Furthermore, h ⁻ _p = (H ₁ (θ _p ),..., H _M (θ _p )) ^T (where H _m (θ _p ) represents the directivity of the microphone m with respect to the sound source p at the angle θ _p ) _{^{, θ - = (θ 1,}} ..., θ P), H - (θ -) = (h - 1, ..., h - P), s - (ω, k, θ -) = (S 1 (ω, k, θ ₁ ),…, S _P (ω, k, θ _P )) ^{T
x - (ω, k) =} prod [H - (θ -), A - (ω)] s - (ω, k, θ -) (9)
Can be written.
Here, prod [A, B] represents the product of each element of the two matrices A and B.

Therefore, if the collected sound signal collected by the directional microphone is considered to have been subjected to linear filtering at the stage of sound collection, the enhancement signal is
^{y - (ω, k) =} x - (ω, k) (10)
It is.

Hereinafter, this embodiment will be described.
<Sound Collection Device 100 according to First Embodiment>
FIG. 7 is a functional block diagram of the sound collecting apparatus 100 according to the first embodiment, and FIG. 8 shows a processing flow thereof.

The sound collection device 100 receives an analog sound collection signal x ₁ (a),..., X _M (a) of a microphone array composed of M microphones m, and emphasizes a sound emitted from the main beam region l (el). reproduction signal z ₁ (t) that, ..., and outputs a z _L (t). M and L are each an integer of 2 or more. a and t are continuous time and an index indicating the continuous time, respectively, L represents the number of main beams, and m = 1, 2,..., M, l = 1, 2,. The sound collection device 100 forms L main beam regions each having L directions as target directions in a microphone array, and collects sound for each main beam region. When the sound collection device 100 detects the presence of a sound source in a sub beam region whose target direction is a superposition region of two adjacent main beam regions in a state where L main beam regions are formed, It is determined that the sound source is closer to the sound source by determining which of the two main beam regions is closer based on the feature amount of the enhancement signal that emphasizes the sound of the main beam region and the sub beam region. It is assumed that the signal from the sound source is included in the main beam region side, and adjustment is performed so as to suppress the sound of the sound source included in the signal on the main beam region side determined not to be close to the sound source. For example, such adjustment is performed in the beam reforming unit 140 and the filter application unit 150 described later.

  Note that each of the L main beam regions is formed so as to pick up sound coming toward the center of the microphone array (see FIG. 2). The number of sub-beam regions is set to L or more, and each sub-beam region is further subdivided into N (N ≧ 1) sub-beam regions.

  The sound collection device 100 includes an input unit 110, a frequency domain conversion unit 120, a beam forming unit 130, a beam reforming unit 140, a filter application unit 150, and a time domain conversion unit 160.

Although the target direction θ _{l, 0} of the main beam is arbitrary, it is assumed that it is arranged at an equal angle of 360 ° for ease of explanation. For example, if the number of main beams is L = 6, the interval θ _{l + 1,0} −θ _{l, 0} between the l-th main beam and the l + 1-th main beam is 60 °.

  The number of sub-beams for one set (6 in this example) of the main beam is arbitrary, but the sub-beams are released between the main beams (in other words, the overlap region formed by the two main beam regions). Therefore, it is desirable to prepare L or more sub-beams. However, since the inverse matrix calculation of power spectral density (PSD) estimation, which will be described later, becomes unstable as the number of sub-beams in one set increases, the number of sub-beams in one set is preferably L. In the present embodiment, a description will be given assuming that L sub-beams are set for L main beams.

  Further, in the present embodiment, one sub-beam region is further subdivided and divided into N sub-beam regions. The number of divisions N is also arbitrary, and can be set to N = 3, for example. Note that one sub-beam region before being divided into N is also referred to as a subset region for convenience.

などに設定できる。 The target direction θ _{l, n} of the sub beam in the sub beam region is also arbitrary. If they are arranged at equal intervals between adjacent main beams, the target direction θ _{l, n} of the n-th sub-beam region in the subset region between the l-th main beam region and the l + 1-th main beam region Is

Etc. can be set.

Hereinafter, the processing content of each part is demonstrated.
<Input unit 110>
The input unit 110 receives M analog sound pickup signals x _m (a) picked up by M microphones m, performs AD conversion on these values, and picks up digital sound pickup corresponding to each microphone m. The signal is converted to a signal x _m (t) (S110) and output to the frequency domain converter 120. The directivity of the M microphones m used here is not limited, and for example, an omnidirectional microphone or a unidirectional microphone can be used.

<Frequency domain transform unit 120>
The frequency domain transform unit 120 receives M digital sound pickup signals x _m (t), and Q points of the digital sound pickup signals x _m ((k−1) Q + 1), x _m (( k-1) Q + 2), ..., x _m (kQ) are stored in a buffer and converted into a frequency-domain sound pickup signal X _m (ω, k) by a method such as discrete Fourier transform (S120), and beam formation is performed. To the unit 130. For example, Q can be set to 256 or 512 in the case of 16 KHz sampling.

<Beam forming unit 130>
The beam forming unit 130 receives M frequency domain sound pickup signals X _m (ω, k), and the main beam target direction {θ _{l, 0} | lε1,..., L} and the sub-beam target direction { According to θ _{l, n} | l∈1,…, L | n∈1,…, N}, the winner gains G ₀ (ω, k, l) and G _n (ω, k, l) (n = 1,2 ,..., N) are calculated and output to the beam reforming unit 140. Note that the main beam target direction {θ _{l, 0} | l∈1, ..., L} and the sub-beam target direction {θ _{l, n} | l∈1, ..., L | n∈1, ..., N} Are set in advance. For example, it may be set as a default value, or may be set by the user of the sound collection device 100. The Wiener gain is a gain used in the Wiener filter. The Wiener filter is a linear filter that minimizes the mean square error between the original sound source signal and the estimated sound source signal on the assumption that there is no correlation between speech and noise. In the present embodiment, the beam forming unit 130 obtains a Wiener filter using the estimated power spectral density (PSD) values of the main beam region and the sub beam region, the beam reforming unit 140 corrects the Wiener filter, The overlapping area of the beam area should not exist.

When using an omnidirectional microphone, the linear filter coefficient W ^- ₀ (ω) (L × M matrix with W _{0_ (m, l)} (ω) as an element) that emphasizes the target direction of the main beam and the sub beam Using the linear filter coefficient W ^- _n (ω) (L × M matrix with W _{n_ (m, l)} (ω) as an element) to emphasize the target direction, the enhancement signal Y ₀ (ω, k, l) and Find and output Y _n (ω, k, l).
_{^{Y - 0 (ω, k)}} = W - 0 (ω) x - (ω, k) (22)
Y ^- ₀ (ω, k) = (Y ^- ₀ (ω, k, 1), Y ^- ₀ (ω, k, 2),…, Y ^- ₀ (ω, k, L))
_{^{Y - n (ω, k)}} = W - n (ω) x - (ω, k) (23)
Y ^- _n (ω, k) = (Y ^- _n (ω, k, 1), Y ^- _n (ω, k, 2),…, Y ^- _n (ω, k, L))

Also, when collecting sound of one main beam region with one directional microphone, M = L, m = l, and Y ₀ (ω, k, l) = X without using a linear filter. Output as _l (ω, k), Y _n (ω, k, l) = X _l (ω, k). That is, when a single directional microphone picks up sound in one main beam region, the output of the directional microphone corresponds to an enhancement signal that emphasizes the sound in the main beam region, and an enhancement signal that enhances the sound in the sub beam region. Do not need. The reason will be described later. For convenience, even if a single directional microphone picks up sound in one main beam region, Y _n (ω, k, l) = X _l (ω, k) is emphasized in the sub-beam region. This is called an enhancement signal. The process of enhancing at least the sound in the main beam region corresponds to beam forming (S130).

<Beam reforming unit 140>
The beam reformer 140 includes L winner gains G ₀ (ω, k, l) and enhancement signals Y ₀ (ω, k, l) corresponding to the main beam, and L × N winners corresponding to the sub beams. The gain G _n (ω, k, l) and the enhancement signal Y _n (ω, k, l) are received, and these values are linked to divide the sound collection area into L main beam areas with no overlap area. L winner gains G (ω, k, l) to be obtained are obtained (S140) and output together with L enhancement signals Y ₀ (ω, k, l) corresponding to the main beam.

<Filter application unit 150>
The filter application unit 150 receives L enhancement signals Y ₀ (ω, k, l) corresponding to the main beam and L winner gains G (ω, k, l), and calculates the product of these values. The obtained L reproduction signals Z _l (ω, k) = Y ₀ (ω, k, l) × G (ω, k, l) are calculated (S 150) and output to the time domain conversion unit 160. The reproduction signal Z _l (ω, k) is a narrow-directional audio signal corresponding to the main beam region l, and is a signal that emphasizes the sound emitted from the main beam region l (L). The difference between the reproduction signal Z _l (ω, k) and the enhancement signal Y ₀ (ω, k, l) is between Y ₀ (ω, k, l) and Y ₀ (ω, k, l + 1). Is that there is no overlap area between the reproduction signals Z _l (ω, k) and Z _{l + 1} (ω, k). More precisely, when a sound source is present in the overlap region between the main beam region l and the main beam region l + 1, the Wiener gain G (ω, k) is added to the enhancement signal Y ₀ (ω, k, l). , l) by multiplying the emphasis signal Y ₀ (ω, k, l + 1) by the Wiener gain G (ω, k, l + 1), the sound source exists in one main beam region, and the other Signal processing is performed so as not to exist in the main beam region. By such signal processing, it is assumed that there is no overlapping region between the reproduction signals Z _l (ω, k) and Z _{l + 1} (ω, k).

<Time domain conversion unit 160>
The time domain transform unit 160 transforms the L frequency domain reproduction signals Z _l (ω, k) into the time domain using a technique such as inverse discrete Fourier transform (S160), and L time domain reproduction signals z _l. Find (t) and output.

<Effect>
With such a configuration, it is determined whether or not a sound source exists in the overlap region without using the similarity, and when a sound source exists in the overlap region, it is obtained from any beam region that forms the overlap region. It is possible to perform processing for suppressing the sensitivity to the sound source existing in the overlap region on the signal.

<Modification>
The point of this embodiment is that there is no overlap region between the reproduction signals Z _l (ω, k) and Z _{l + 1} (ω, k), and the Wiener filter G (ω, k) for that purpose. , l). Therefore, the sound collection device 100 only needs to include at least the beam forming unit 130 and the beam reshaping unit 140, and other configurations may be provided as separate devices. For example, the collected sound signal X _m (ω, k) in the frequency domain may be input and the Wiener filter G (ω, k, l) may be output.

  The intervals between the main beams and the sub beams do not need to be equal. Further, the number of subset areas need not be L or more, and the number of sub-beam areas included in each subset area may be changed. However, the inverse matrix calculation of power spectral density (PSD) estimation described later becomes unstable.

  In the present embodiment, the sound collection region is 360 ° with respect to the center of the microphone array, but any sound collection region may be used. Any sound collection area may be used as long as the adjacent main beam areas form a superposition area and the sub-beam area is formed in the superposition area.

<Second embodiment>
In the second embodiment of the present invention, the functional configuration of the beam forming unit 130 of the first embodiment is embodied. 9 shows a functional block diagram of the beam forming unit 130, and FIG. 10 shows an example of a flowchart thereof.

  The beam forming unit 130 includes N + 1 winner gain calculating units 131-q and a memory 134. Here, q = 0, 1,..., N, and q = 0 is an index indicating the main beam.

The beam forming unit 130 forms a narrow-directed speech enhancement beam independently for each of the target directions of the main beam and the sub beam using the technique of Reference 2.
(Reference 2): Y. Hioka, K. Furuya, K. Kobayashi, K. Niwa, Y. Haneda, "Underdetermined sound source separation using power spectrum density estimated by combination of directivity gain", IEEE Transactions on Audio, Speech, and Language Processing, 2013, Vol.21-6, pp.1240-1250.

  Specifically, a Wiener gain (time frequency mask) calculation based on an estimated value of a power spectrum density (hereinafter also referred to as “PSD”) of an enhanced signal that emphasizes the sound of each beam region is performed.

The winner gain calculation unit 131-q is an implementation of Reference 2. Reference 2, the sound source s ^- (ω, k) to estimate the PSD of enhanced signal y ^- (ω, k) to the appropriate Wiener gain ^{G - (ω, k) =} (G (ω, k, 1), ..., G (ω, k, L)) are calculated, and post-processing is performed to further enhance speech. A detailed algorithm is given at the end of this section.

<Winner Gain Calculation Unit 131-q>
FIG. 11 shows a configuration example of the winner gain calculation unit 131-q.

  The winner gain calculation unit 131-q includes a linear filtering unit 131-q-1, a PSD estimation unit 131-q-2, a square root calculation unit 131-q-3, and a gain calculation unit 131-q-4.

The winner gain calculation unit 131-q receives the M frequency-range collected sound signals X _m (ω, k), and uses these values to determine the winner gain G _q (ω, k, l) and the enhancement signal. _{Y q (ω, k, l} ) and the sound source of _{PSDφ S_q (ω, k, l} ) ( however, A_B subscript represents. a a _B) as taking the square root of S _q (ω, k, l) and output simultaneously. The enhancement signal G _q (ω, k, l) forms L main beam regions and L × N sub beam regions using M omnidirectional microphones, and generates sound in each beam region. Is a signal after linear filtering that is calculated in the process of calculating the Wiener gain G _q (ω, k, l), and the sound of one main beam region is collected by one directional microphone. If so, it is the output value of the directional microphone. As described above, q = 0, 1,..., N, and n = 1, 2,.

Reference Document 2 will be briefly described below. When sound is collected using an omnidirectional microphone (S131-q-0), first, the winner gain calculation unit 131-q loads (takes out) the linear filter coefficient W ^- _q (ω) from the memory 134. Then, linear filtering unit 131-q-1 is the following equation, the collected signal x ^- (ω, k) and each beam of the linear filter coefficient W ^- taking the product of _q (omega), after linear filtering narrow A directivity enhancement signal Y _q (ω, k) is obtained (S131-q-1).
_{Y q (ω, k) =} W - q (ω) x - (ω, k) (31)
In addition, when the sound of one main beam area is picked up by one directional microphone (S131-q-0), this process S131-q-1 may not be performed. As described above, Y _q (ω, k, l) = X _l (ω, k) may be output.

From the formulation of the above-mentioned array collected sound signal enhancement technique, enhancement audio y after linear filtering ^{- (ω,} k) is the sound source s ^{- (ω,} k), transfer characteristic A ^- (omega), the collected signal x ^- ( Using ω, k) and the linear filter coefficient W ⁻ (ω), it can be described as follows (see equation (6)). ^{y - (ω, k) =} W - (ω) x - (ω, k) = W - (ω) A - (ω) s - (ω, k) = D - (ω) s - (ω, k ) (32)
^{However, D - (ω) = W} - (ω) A - (ω)
In the case of picking up the sound of one main beam region in one directional ^{microphone, y - (ω, k)} = x - (ω, k) ^{a, y - (ω, k)} = D - If (ω) s ⁻ (ω, k), then D ⁻ (ω) = prod [H ⁻ (θ ⁻ ), A ⁻ (ω)] (see equations (9) and (10)).

と書くことができる。ここでE[]は期待値演算、上付き文字の＊は複素共役を表す。さらに、Φ^- _y(ω,k)=(φ_y(ω,k,1),…,φ_y(ω,k,L))^T、Φ^- _S(ω,k)=(φ_S(ω,k,1),…,φ_S(ω,k,P))^T、D^-(ω)を|D_l,p(ω)|²を成分とするL×P行列とすることで、
Φ^- _y(ω,k)=D^-(ω)Φ^- _S(ω,k) (34)
と行列形式で記述できる。したがって、音源s^-(ω,k)のPSDは、
Φ^- _S(ω,k)=D^-+(ω)Φ^- _y(ω,k) (35)
と求めることができる（Ｓ１３１−ｑ−２）。ここで上付き文字の＋は擬似逆行列を示す。また、D^-(ω)が正則行列であれば(L=Pであり、メインビームの個数と音源の個数とが同じ場合であれば)、擬似逆行列を逆行列に置き換えてもよい。 Here, PSDφ _y (ω, k, l) = E [Y (ω, k, l) Y (ω, k, l) ^* ] of the narrow-directed emphasized speech y ⁻ (ω, k) is P pieces By assuming that the sound sources are independent,

Can be written. Here, E [] represents an expected value calculation, and the superscript * represents a complex conjugate. Furthermore, Φ ^- _y (ω, k) = (φ _y (ω, k, 1),…, φ _y (ω, k, L)) ^T , Φ ^- _S (ω, k) = (φ _S (ω , k, 1), ..., φ _S (ω, k, P)) ^T , D ⁻ (ω) is an L × P matrix with | D _{l, p} (ω) | ² as components,
_{^{Φ - y (ω, k)}} = D - (ω) Φ - S (ω, k) (34)
And can be described in matrix form. Therefore, the sound source s ^- (ω, k) PSD of,
Φ ^- _S (ω, k) = D- ⁺ (ω) Φ ^- _y (ω, k) (35)
(S131-q-2). Here, the superscript + indicates a pseudo inverse matrix. Further, if D ⁻ (ω) is a regular matrix (if L = P and the number of main beams and the number of sound sources are the same), the pseudo inverse matrix may be replaced with an inverse matrix.

と計算できる（Ｓ１３１−ｑ−４）。 Therefore, the appropriate gain for the target direction is as follows: i∈θ _l is used as the sound source number whose arrival direction coincides with the target direction θ _l .

(S131-q-4).

としてウィナーゲインを計算する（Ｓ１３１−ｑ−４）。 However, in many cases in the real environment, transmitted arrival direction of the sound source characteristics A ^- (omega) is unknown. Therefore At that time, transfer characteristics A ^- a (omega), the approximation is calculated using the array manifold vector representing the phase difference of the propagating wave from the target direction theta _l to the microphone position. Also, assuming that L = P, the sound source signal s _l (t) is assumed to arrive from the target direction θ _l ,

As a result, the winner gain is calculated (S131-q-4).

If the noise level of the sound source other than the desired sound source is high, noise resistance processing may be performed at the time of calculating the winner gain, as in Reference 3.
(Reference 3) K. Niwa, Y. Hioka, K. Kobayashi, “Post-filter design for speech enhancement in various noisy environments”, 14th International Workshop on Acoustic Signal Enhancement (IWAENC), 2014, pp. 35-39.

(Linear filtering unit 131-q-1)
Therefore, when sound is collected using an omnidirectional microphone (S131-q-0), in other words, using M omnidirectional microphones, L main beam regions and L × N forming a sub-beam region, to pick up the sound of each beam region, linear filtering unit 131-q-1 is the linear filter coefficient W ^- _q (ω) and the collected sound signal x ^{- (ω,} k) and the Then, linear filtering is performed according to equation (31) (S131−q−1), and the enhancement signal Y _q (ω, k) = (Y _q (ω, k, 1), Y _q (ω, k, 2) ,..., Y _q (ω, k, L)) are calculated and output.

When sound is collected using a directional microphone (S131-q-0), in other words, when sound of one main beam region is collected by one directional microphone, this processing S131-q-1 is performed. It does not have to be. M = L, m = l, and the linear filtering unit 131-q-1 may output Y _q (ω, k, l) = X _l (ω, k).

(PSD estimation unit 131-q-2)
The PSD estimation unit 131-q-2 receives the enhancement signal Y _q (ω, k, l), estimates the PSD according to Equation (35) (S131-q-2), and estimates the value φ _{S_q} (ω, k , l). However, assuming L = P,
Φ _{S_q} (ω, k) = D ^- _q ⁺ (ω) Φ _{y_q} (ω, k)
Φ _{S_q} (ω, k) = (φ _{S_q} (ω, k, 1),…, φ _{S_q} (ω, k, L)) ^T
Φ ^- _{y_q} (ω, k) = (φ _{y_q} (ω, k, 1),…, φ _{y_q} (ω, k, L)) ^T
φ _{y_q} (ω, k, l) = E [Y _q (ω, k, l) Y _q (ω, k, l) ^* ]
It is.

Incidentally, the matrix in equation (35) D ^- (ω), when picked up by using a non-directional microphones, transfer characteristic A ^- (omega) and the linear filter coefficient W ^- with _q (omega) ,
_{^{D - q (ω) = W}} - q (ω) A - (ω)
And it represents, when picked up by using the directional microphones, transfer characteristic A ^- (omega) and directional H ^- (θ ^- _q) with,
_{^{D - q (ω) = prod}} [H - (θ - q), A - (ω)]
It can be expressed as. θ ^- _q = (θ _{1_q} , ..., θ _{L_q} ), and θ _{l_q} is the angle formed by the target direction of the n-th sub-beam region of the l-th subset region with respect to the reference direction when q = n. Represent. When q = 0, it represents the angle formed by the target direction of the l-th main beam region with respect to the reference direction. When sound is collected using a directional microphone, the enhancement signal Y _q (ω, k, l) is Y _q (ω, k, l) = X _l (ω, k), and the sound in the sub-beam region is Although it cannot be said that the signal is an enhanced signal, the estimated value φ _{S_q} (ω of PSD for the enhanced signal Y _q (ω, k, l) that emphasizes the sound in the sub-beam region using such a matrix D ⁻ _q (ω). , k, l). Furthermore, the square root calculator 131-q-3 and the gain calculating unit 131-q-4 will be described later, the estimate _{φ S_q (ω, k, l} ) for processing with emphasis that emphasizes the sound of the sub-beam region The value corresponding to the signal Y _q (ω, k, l) (the square root S _q (ω, k, l) and the winner gain G _q (ω, k, l) of the estimated value φ _{S_q} (ω, k, l)) Can be obtained. Estimates _{φ S_q (ω, k, l} ) and its square root _{S q (ω, k, l} ) and enhanced signal _{Y q (ω, k, l} ) also referred to as feature amount corresponding to.

(Square root calculation unit 131-q-3)
The square root calculation unit 131-q-3 receives the estimated value φ _{S_q} (ω, k, l) of PSD, and the square root S _q (ω, k, l) = √ (φ _{S_q} (ω, k, l)) Is calculated (S131-q-3) and output.

言い換えると、N個のゲイン計算部１３１−ｑ−４は、L個のメインビーム領域ごとの音を強調した強調信号Y₀(ω,k,l)およびL×N個のサブビーム領域ごとの音を強調した強調信号Y_q(ω,k,l)について音源の存在を検出するために用いるフィルタG^-(ω,k,l)=｛G₀(ω,k,l),G₁(ω,k,l),…,G_N(ω,k,l)｝を形成する。なお、このフィルタG^-(ω,k,l)は、前述の通り、ウィナーフィルタである。 (Gain calculator 131-q-4)
The gain calculation unit 131-q-4 receives the estimated value φ _{S_q} (ω, k, l) of the PSD and calculates the winner gain G _q (ω, k, l) by using the equation (36) or the equation (37). (S131-q-4) and output.

In other words, the N gain calculators 131-q-4 perform the enhancement signal Y ₀ (ω, k, l) that emphasizes the sound for each of the L main beam regions and the sound for each of the L × N sub beam regions. enhancement signal emphasizing the _{Y q (ω, k, l} ) filter used to detect the presence of a sound source for ^{G - (ω, k, l} ) = {G 0 (ω, k, l), G 1 (ω , k, l),..., G _N (ω, k, l)}. Incidentally, the filter ^{G - (ω, k, l} ) , as described above, is a Wiener filter.

<Effect>
With such a configuration, the same effect as that of the first embodiment can be obtained.

<Modification>
When performing linear filtering, processing is performed on the assumption that M microphones are omnidirectional, but each of the M microphones may have directivity or omnidirectionality. Alternatively, a directional microphone and an omnidirectional microphone may be mixed. Even when a microphone having directivity is used, by performing linear filtering, a beam can be formed and sounds in the main beam region and the sub beam region can be collected. Note that similar modifications are possible in the first embodiment.

<Third embodiment>
In the third embodiment of the present invention, the functional configuration of the beam reshaping unit of the first embodiment is embodied. FIG. 12 is a functional block diagram of the beam reshaping unit 140, and FIG. 13 shows an example of the processing flow.

The beam reforming unit 140 includes an enhancement signal Y _q (ω, k, l), a winner gain G _q (ω, k, l), and an estimated value φ _{S_q} (ω, k, l) of each PSD of the beam region. ) Square root S _q (ω, k, l) (where q = 0, 1,..., N, l = 1, 2,..., L) and superimposing two adjacent main beam regions When it is detected that a sound source is present in the sub-beam region with the region as the target direction, the winner gain G ₀ (ω, k, l) or G ₀ (ω, k, l + 1) in the main beam region is suppressed. However, if it is not detected that a sound source is present in the sub-beam region, the main beam region Y ₀ (ω, k, l) is not suppressed without suppressing the main beam region's winner gain G ₀ (ω, k, l). , l). An example of realizing the following processing will be described.

  The beam reshaping unit 140 includes a sound source direction detecting unit 141, a gain suppressing unit 142, and a gain correcting unit 143.

(Sound source direction detection unit 141)
The sound source direction detection unit 141 compares the two main beams and the outputs of N sub-beams between them as shown in FIG. 6, and detects the sound source direction.

The emphasized speech and winner gain of the two main beams are respectively Y ₀ (ω, k, l), G ₀ (ω, k, l) and Y ₀ (ω, k, l + 1), G ₀ (ω, k, l + 1), and the sound source direction detection unit 141 uses Y ₀ (ω, k, l), G ₀ (ω, k, l), Y ₀ (ω, k, l + 1), G ₀ ( ω, k, l + 1) is received and the absolute value of the product of the emphasized speech and the Wiener gain is calculated (S141a). For convenience, | Y ₀ (ω, k, l) × G ₀ (ω, k, l) | is called (a), and | Y ₀ (ω, k, l + 1) × G ₀ (ω, k, l + 1) | is called (b). In FIG. 12, only Y ₀ (ω, k, l) and G ₀ (ω, k, l) are received, but since Y = 1, 2,..., L are received, Y ₀ (ω, k, l + 1), G ₀ (ω, k, l + 1) can also be received. When l = L, the first Y ₀ (ω, k, 1), G ₀ (ω, k, 1) is used as the l + 1st emphasized speech and the winner gain. Also in the following description, when l = L (in other words, l + 1> L), l + 1 is read as 1, and is calculated.
(a) = | Y ₀ (ω, k, l) × G ₀ (ω, k, l) | (41)
(b) = | Y ₀ (ω, k, l + 1) × G ₀ (ω, k, l + 1) | (42)

Next, the emphasized speech and Wiener gain of the N sub-beams existing between the l-th main beam and the l + 1-th main beam are respectively _expressed as Y _n (ω, k, l), G _n (ω, k , l). The sound source direction detection unit 141 receives N pieces of Y _n (ω, k, l) and G _n (ω, k, l), respectively, and calculates the absolute value of the product of the enhanced speech and the Wiener gain in the same manner as the main beam. (S141b). For convenience, | Y _n (ω, k, l) × G _n (ω, k, l) | is called (c, n).
(c, n) = | Y _n (ω, k, l) × G _n (ω, k, l) | (43)

  Then, a search is made for the largest value among (a), (b) and N (c, n) (S141c). If (a) or (b) is maximum, it is determined that there is a sound source in the target direction of the main beam, and if any of (c, n) is maximum, it is determined that there is a sound source in the target direction of the sub beam. To do. Further, for the subsequent gain correction unit 143, when the sub beam is the maximum, which sub beam is the maximum is stored. Specifically, the index n of the sub beam that has been maximized is stored in the variable F1 (l) (F1 (l) ← n). If any of the main beams is maximum, F1 (l) ← 0. The sound source direction detection unit 141 outputs the obtained F1 (l).

  In other words, the beam reshaping unit 140 compares the signal power calculated for each filter, and determines whether the region where the signal power is maximum is the sub-beam region or the main beam region.

(Gain suppression unit 142)
The gain suppression unit 142 includes F1 (l), Y ₀ (ω, k, l), G ₀ (ω, k, l), Y ₀ (ω, k, l + 1), G ₀ (ω, k, l + 1) is received, and if there is a sound source in the target direction of the sub-beam (if F1 (l) ≠ 0), the winner gain of the l-th main beam or the l + 1-th main beam is suppressed. In order to determine which side (a) or (b) the sound source on the sub beam exists on, the values of (a) and (b) are compared (S142a). By this process, it is determined which sound source estimated to be present in the sub-beam region is closer to the adjacent main beam region. Assuming that the sound sources in all regions are sparse and independent in the frequency direction, Y ₀ (ω, k, l) and Y ₀ (ω, k, l + 1) contain information for only one sound source. If (a) is large, the sound source is near the target direction of the l-th main beam, and if (b) is large, the sound source is near the target direction of the l + 1-th main beam. Conceivable.

Based on this, if (a) is large, G ₀ (ω, k, l + 1) is set to a small value g _sp smaller than the original value (G ₀ (ω, k, l + 1) ← g _{If sp} , S142b) and (b) are large, G ₀ (ω, k, l) is set to a small value g _sp smaller than the original value (G ₀ (ω, k, l) ← g _sp , S142c ). g _sp can be set to 0.0 or 0.05, for example. The process of setting this G ₀ (ω, k, l + 1) or G ₀ (ω, k, l) to a small value is G ₀ (ω, k, l + 1) or G ₀ (ω, k, This corresponds to the processing of suppressing l). If the value _gsp is sufficiently small, the value _gsp may be larger than the original value of G ₀ (ω, k, l) or G ₀ (ω, k, l + 1). This is because if the value _gsp is sufficiently small (for example, less than 0.05), the output is hardly affected even if it is set larger than the original value.

  The above-described processing is performed on the target directions of all main beams (S140a, S140b, S140c).

  An algorithm that absorbs fluctuations in the sound source position can be added to the comparison process described above. For example, it is assumed that the target directions of the l-th main beam and the l + 1-th main beam are 0 ° and 60 °, respectively, and that a sound source such as a human exists near 30 °. This sound source may have temporal fluctuations, for example, when the sound source position is 29 ° at time k and the sound source position is 31 ° at time k + 1 due to factors such as neck rotation and re-sitting. .

In such a situation, processing such as suppressing G ₀ (ω, k, l) at time k and suppressing G ₀ (ω, k + 1, l + 1) at time k + 1 occurs, and output In some cases, the continuity of time may not be guaranteed. In order to prevent this, an algorithm for absorbing slight fluctuations in the sound source position may be additionally implemented.
As an example of the algorithm, the following conditions
C.1 The value of (a) is less than g _th at time k-1.
C.2 The value of (a) is greater than or equal to g _th at time k.
C.3 (a)> (b) at time k.
If the time is satisfied,
As long as D.1 (a) is greater than or equal to g _th
D.2 Even if (b)> (a), if (b) is not more than r _th times (a),
A process such as suppressing G ₀ (ω, k, l + 1) can be considered. This process can be handled even when (b) reacts first by reading (a) and (b) interchangeably. The values of g _th and r _th are preferably determined experimentally. For example, g _th can be set to 0.2, r _th can be set to 1.4, and the like. Also, for voice etc., it may be lower than g _th locally due to prompting sound, etc., so condition D.1
D.1 t _th (ms) or more, if not continuously less than g _th
It may be changed. The value of t _th can be set to 500, for example.

  In this way, the gain suppression unit 142 once determines that the main beam region that is not close to the sound source is not close to the sound source, even if it is determined that the main beam region is close to the sound source. Then, adjustment is performed so as to suppress the sound of the sound source included in the signal on the main beam region side determined not to be close to the sound source.

When there is a sound source in the target direction of the sub-beam, the gain suppression unit 142 suppresses the received winner gain and the received winner gain (that is, G ₀ (ω, k, l) = g _sp and G ₀ (ω, k, l +1) or G ₀ (ω, k, l) and G ₀ (ω, k, l + 1) = g _sp ), and if there is a sound source in the target direction of the main beam, the received winner The gain is output as it is (G ₀ (ω, k, l) and G ₀ (ω, k, l + 1)). Note that the winner gain G ₀ (ω, k, l) corresponding to the l-th main beam is related to the winner gain G ₀ (ω, k, l-1) corresponding to the (l-1) -th main beam. In some cases, it may be suppressed, or may be suppressed in relation to the Wiener gain G ₀ (ω, k, l + 1) corresponding to the l + 1-th main beam.

(Gain correction unit 143)
Finally, the gain correction unit 143 _calculates the square root S _q (ω, k, l) of F1 (l), the winner gain G ₀ (ω, k, l), and the estimated PSD φ _{S_q} (ω, k, l) of the sound source. l) (output value of the square root calculation unit 131-q-3) is received, a gain correction coefficient λ (ω, k) or Λ ⁻ (ω, k) described later is calculated and applied, and the magnitude of the entire gain is calculated. Adjust (S143). For processing to reduce the Wiener gain in the gain suppression unit 142, when the still time-frequency mask, the total volume of the reproduced signal z _l (t) is smaller than the total amount of the sound volume of the sound source signals s _p (t) There is a possibility. In addition, since the present embodiment performs processing independently for the time frame, there is a possibility that temporal continuity is lost from the volume of the reproduction signal. In order to solve these problems, the overall gain is adjusted.

  360 ° (omnidirectional) is divided into L regions, and the amplitude spectrum of the sound source existing in the l-th region is | S (ω, k, l) |. In addition, the amplitude spectrum of a signal in which only the speech existing in the l-th region is emphasized is | Z (ω, k, l) |. The purpose of this technology is to realize narrow-directional speech enhancement such that | S (ω, k, l) | = | Z (ω, k, l) |.

Here, the amplitude spectrum vector of the enhancement signal is Y ⁻ _{ω, k} = (| Y ₀ (ω, k, 1) |,…, | Y ₀ (ω, k, L) |) ^T , Diagonal matrix with vectors g ~ (ω, k) = (G ₀ (ω, k, 1), ..., G ₀ (ω, k, L)) and g ~ (ω, k) as diagonal components the when the G ~ (ω, k), Wiener filtering the signal after the amplitude spectrum _{^{Z - ω, k = (|}} Z - (ω, k, 1) |, ..., | Z - (ω, k, L ) |) ^T is
Z ^- _{ω, k} = G ~ (ω, k) Y ^- _{ω, k} (44)
Can be described.

Therefore, in order to realize narrow-directional speech enhancement such that | S (ω, k, l) | = | Z (ω, k, l) |, S ⁻ _{ω, k} = (S _{F1 (1)} (ω , k, 1), ..., S F1 (L) (ω, k, as L)), the objective function _{^{J = | S - ω, k}} -G - (ω, k) Y - ω, k | minimum ² The gain G˜ (ω, k) may be adjusted to the gain G ⁻ (ω, k) so that The value of S _{F1 (l)} (ω, k, l) can be obtained from F1 (l) and the square root S _q (ω, k, l).

If this problem is corrected by amplifying the suppressed gain at the same level in all regions as a method of adjusting the gain,
^{G - (ω, k) =} λ (ω, k) G ~ (ω, k) (45)
If the correction to amplify at different levels in each region is performed,
^{G - (ω, k) =} Λ - (ω, k) G ~ (ω, k) (46)
To become diagonal matrix Λ ^- (ω, k) results in a problem of finding a.

で計算できる。ここでdiag[・]は[・]内の行列の非対角成分を０にする処理である。詳細な導出は、以下の通りである。 Λ (ω, k) and Λ ⁻ (ω, k) satisfying this are obtained by partial differentiation of the objective function J with each parameter,

It can be calculated with Here, diag [•] is processing for setting the non-diagonal component of the matrix in [•] to 0. Detailed derivation is as follows.

のように目的関数をλ(ω,k)で偏微分して0と置くと、
λ(ω,k)Y^- _ω,k ^TG~^T(ω,k)G~(ω,k)Y^- _ω,k=Y^- _ω,k ^TG~^T(ω,k)S^- _ω,k

となる。なお、行列Gは対角行列であり、G~(ω,k)=G~^T(ω,k)なので、実装上は、計算量を減らすために、

としてもよい。 (Derivation of λ (ω, k))
The objective function J = | S - ω, k -G - (ω, k) Y - ω, k | based 2 on the formula (45),
J = (S - ω, k -λ (ω, k) G ~ (ω, k) Y - ω, k ) T (S - ω, k -λ (ω, k) G ~ (ω, k) Y - ω, k) (49)
And the formula deformed,

If the objective function is partially differentiated by λ (ω, k) and set to 0,
λ (ω, k) Y ^- _{ω, k} ^T G ~ ^T (ω, k) G ~ (ω, k) Y ^- _{ω, k} = Y ^- _{ω, k} ^T G ~ ^T (ω, k) S ^- _{ω , k}

It becomes. Note that the matrix G is a diagonal matrix, and G ~ (ω, k) = G ~ ^T (ω, k).

It is good.

のように目的関数をΛ^-(ω,k)で偏微分して0と置くと、
Y^- _ω,k ^TG~^T(ω,k)Λ^-(ω,k)G~(ω,k)Y^- _ω,k=Y^- _ω,k ^TG~^T(ω,k)S^- _ω,k
Λ^-(ω,k)G~(ω,k)Y^- _ω,kY^- _ω,k ^T=S^- _ω,kY^- _ω,k ^T
Λ^-(ω,k)=S^- _ω,kY^- _ω,k ^T(G~(ω,k)Y^- _ω,kY^- _ω,k ^T)^-1 (53)
となる。ここでΛ^-(ω,k)はそれぞれの方向のメインビームのゲインを修正する係数なので、非対角成分を０にする処理（diag[・]）を行う。 (Λ ^- (ω, k) the derivation of)
The objective function _{^{J = | S - ω, k}} -G - (ω, k) Y - ω, k | based ² in equation (46),
_{^{J = (S - ω, k}} -Λ - (ω, k) G ~ (ω, k) Y - ω, k) T (S - ω, k -Λ - (ω, k) G ~ (ω, k ) Y ^- _{ω, k} ) (51)
And the formula deformed,

The objective function Λ as ^{- (ω,} k) 0 and when put to partial differential in,
_{^{Y - ω, k T G ~}} T (ω, k) Λ - (ω, k) G ~ (ω, k) Y - ω, k = Y - ω, k T G ~ T (ω, k) S - _{ω, k}
^{Λ - (ω, k) G} ~ (ω, k) Y - ω, k Y - ω, k T = S - ω, k Y - ω, k T
^{Λ - (ω, k) =} S - ω, k Y - ω, k T (G ~ (ω, k) Y - ω, k Y - ω, k T) -1 (53)
It becomes. Here, since Λ ⁻ (ω, k) is a coefficient for correcting the gain of the main beam in each direction, a process (diag [•]) for setting the off-diagonal component to 0 is performed.

From the above, the corrected winner gain G (ω, k, l) for the l-th beam is
G (ω, k, l) = λ (ω, k) G ~ (ω, k, l) (54)
Or
G (ω, k, l) = Λ ^- _ll (ω, k) G ~ (ω, k, l) (55)
It can be calculated with Here Λ ^- _ll (ω, k) is Λ ^- (ω, k) is the l-th diagonal elements of.

The gain correction unit 143 outputs a corrected winner gain when there is a sound source in the target direction of the sub-beam. If there is a sound source in the target direction of the main beam, the gain correction unit 143 outputs the received winner gain as it is (G (ω, k, l) = G ₀ (ω, k, l)) is output.

  Therefore, the reproduction signal for each main beam area output from the sound collection device 100 is the sum of the powers of the signals in each main beam area after adjustment of the main beam area and the power of the sound collection signals in each main beam area before adjustment. It can be said that the gain correction unit 143 adjusts the gain with respect to the signal after the adjustment of the main beam region so that the error from the sum of the two is minimized.

<Effect>
By setting it as such a structure, the effect similar to 1st embodiment can be acquired.

<Other variations>
The present invention is not limited to the above-described embodiments and modifications. For example, the various processes described above are not only executed in time series according to the description, but may also be executed in parallel or individually as required by the processing capability of the apparatus that executes the processes. In addition, it can change suitably in the range which does not deviate from the meaning of this invention.

<Program and recording medium>
In addition, various processing functions in each device described in the above embodiments and modifications may be realized by a computer. In that case, the processing contents of the functions that each device should have are described by a program. Then, by executing this program on a computer, various processing functions in each of the above devices are realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Further, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its storage unit. When executing the process, this computer reads the program stored in its own storage unit and executes the process according to the read program. As another embodiment of this program, a computer may read a program directly from a portable recording medium and execute processing according to the program. Further, each time a program is transferred from the server computer to the computer, processing according to the received program may be executed sequentially. Also, the program is not transferred from the server computer to the computer, and the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by the execution instruction and result acquisition. It is good. Note that the program includes information provided for processing by the electronic computer and equivalent to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer).

  In addition, although each device is configured by executing a predetermined program on a computer, at least a part of these processing contents may be realized by hardware.

Claims (7)

M and L are each an integer of 2 or more, and L main beam regions each having L directions as target directions are formed by a microphone array composed of M microphones, and sound is generated for each main beam region. A sound collecting device for collecting sound,
When it is detected that a sound source exists in a sub-beam region whose target direction is a superposition region of two adjacent main beam regions in a state where L main beam regions are formed, the sound source is divided into two main beam regions. The signal on the main beam region side that is determined not to be close to the sound source is determined based on the feature amount of the emphasis signal that emphasizes the sound in the main beam region and the sub beam region. Adjust to suppress the sound of the sound source included in
Sound collection device. The sound collection device according to claim 1,
N is any integer of 1 or more, and the L main beam regions are formed so as to collect sounds arriving toward the center of the microphone array, the number of sub beam regions is L or more, and A filter used for subtracting each sub-beam area into N sub-beam areas and detecting the presence of a sound source for the feature quantity of the enhancement signal that emphasizes the sounds in the L main beam areas and the L × N sub-beam areas. Forming and comparing the signal power calculated for each filter to determine whether the region where the signal power is maximum is the sub-beam region or the main beam region,
Sound collection device. The sound collecting device according to claim 1 or 2,
When detecting the presence of a sound source in the main beam region and the sub beam region, a Wiener filter is used in a signal obtained by converting a microphone sound pickup signal into a frequency region
Sound collection device. The sound collecting device according to any one of claims 1 to 3,
The reproduction signal for each main beam area output by the sound collection device is the sum of the power of the signals of each main beam area after adjustment of the main beam area and the sum of the powers of the sound collection signals of each main beam area before adjustment. And adjust the gain for the signal after adjustment of the main beam area so that the error with
Sound collection device. The sound collecting device according to any one of claims 1 to 4,
For the main beam region that has been determined not to be close to the sound source, even if it is determined to be close to the sound source, the main beam that has been determined not to be close to the sound source unless a predetermined condition is satisfied. Adjust so as to suppress the sound of the sound source included in the signal on the region side,
Sound collection device. M and L are each an integer of 2 or more, and L main beam regions each having L directions as target directions are formed by a microphone array composed of M microphones, and sound is generated for each main beam region. A sound collection method for collecting sound,
When it is detected that a sound source exists in a sub-beam region whose target direction is a superposition region of two adjacent main beam regions in a state where L main beam regions are formed, the sound source is divided into two main beam regions. The signal on the main beam region side that is determined not to be close to the sound source is determined based on the feature amount of the emphasis signal that emphasizes the sound in the main beam region and the sub beam region. Adjust to suppress the sound of the sound source included in
Sound collection method.   A program for causing a computer to function as the sound collecting device according to any one of claims 1 to 5.

Images