JP2018120129A - Sound field estimation device, method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technic capable of estimating precisely the sound field in a wider range.SOLUTION: A sound field estimation device comprises: a sound field representation vector estimating unit 120 which estimates a sound field representation vector a(ω)(ω=1,2,...,F) as a vector composed of decomposition coefficients a(ω, q)(ω=1, 2,...,F,q=1,2,...,Q) of mixed waves constituting the sound field, from a frequency domain collection sound signal u(ω,j) (ω=1,2,...,F,j=1,2,...,J) based on the signal collected by the spherical microphone array and a frequency domain collection signal u(ω,j)(ω=1,2,...,F,j=1,2,...,J) based on the signal collected by the additional microphone array group, by using the cost function J(ω)(ω=1,2,...,F); and a sound field calculation unit 130 which calculates the virtual microphone frequency domain collection signal u(ω,r)(ω=1,2,...,F), from the sound field representation vector a(ω)(ω=1,2,...,F) and a virtual microphone position r.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a sound field estimation technique, and more particularly to a technique for estimating a sound field from a collected sound signal collected using a spherical microphone array and an additional microphone.

  In recent years, the number of channels and the number of speakers used for audio playback has increased from 2 to 5.1 and further to 22.1 in order to increase the sense of reality and simultaneously expand the playback area. It is important to measure and estimate the reproduced sound field for evaluation and verification of a reproduction method with an expanded number of channels.

  As such a sound field estimation method, Non-Patent Document 1 proposes a method using a spherical microphone array. In this method, a signal is picked up by a plurality of microphones arranged in a hard spherical spherical microphone array having a baffle, and plane wave decomposition of the sound field is obtained from the multichannel signal, and the sound field is estimated.

B. Rafaely, “Analysis and Design of Spherical Microphone Arrays”, IEEE Trans. Speech Audio Processing, Vol.13, No.1, pp.135-143, Jan. 2005. E. G. Williams, “Fourier Acoustics”, Academic Press, verse 6.10, pp.224-231, 1999.

  In a conventional sound field estimation method using a single spherical microphone array, the sound field measurement points are limited to a spherical surface having a radius of several centimeters to several tens of centimeters. Since the sound field is estimated from the measurement point by extrapolation, there is a possibility that the range in which the accuracy of the estimated sound field can be secured is limited.

  An object of the present invention is to provide a sound field estimation apparatus, method, and program capable of accurately estimating a sound field in a wider range.

In the sound field estimation apparatus according to an aspect of the present invention, the number of microphones on the spherical microphone array is J, and a pair of elevation angle and azimuth angle that specifies the placement position of the microphone is (Θ _j , Φ _j ) (j = 1 , 2, ..., and J), the radius of the spherical microphone array and r _a, the center position of the spherical microphone array as the origin, the number of additional microphones groups microphones and J _c, 3-dimensional coordinate system plane wave model Is a model of a plane wave (q = 1, 2,..., Q) that is incident at an incident angle (θ _q , φ _q ), and the j-th microphone (j = 1, 2,...) On the spherical microphone array. , J) is the plane wave frequency domain observation signal, which is an observation signal in the frequency domain of the plane wave observed by v _s (ω, q, j) (ω is an index representing frequency, ω = 1, 2, ..., F , q = 1, 2, ..., and Q) and then, the j _c microphone in the additional microphone group _{(j c = 1, 2,} ..., observed in J _c) A plane wave frequency domain observed signal is the observation signal in the frequency domain of a plane wave _{v c (ω, q, j} ) (ω = 1, 2, ..., F, q = 1, 2, ..., Q) and, d _s Let (ω, q) (ω = 1, 2, ..., F, q = 1, 2, ..., Q) be the J-dimensional q-plane wave vector defined by the following equation:

Let d _c (ω, q) (ω = 1, 2, ..., F, q = 1, 2, ..., Q) be the J _c- dimensional q-th spherical wave vector defined by the following equation:

Let D (ω) (ω = 1, 2, ..., F) be a (J + J _c ) × Q dictionary matrix defined by the following equation:

Using λ as a regularization parameter, the cost function J (ω) (ω = 1, 2, ..., F) is expressed as the sound field expression vector a (ω) (ω = 1, 2, ..., F) and the decomposition coefficient a _p J (ω) defined by the following equation using (ω, q) (ω = 1, 2, ..., F, q = 1, 2, ..., Q)

The sound field expression vector a (ω) (ω = 1, 2,..., F) is the decomposition coefficient a _p (ω, q) (ω = 1, 2,... F, q = 1, 2,. , Q) to be a Q-dimensional vector defined by

u _s (ω), u _c (ω) (ω = 1, 2,…, F) are added to the frequency domain sound signal vector and J _c dimension of the J-dimensional spherical microphone array defined by the following equations, respectively. As the frequency domain sound pickup signal vector of the microphone group,

Frequency domain sound pickup signal u _s (ω, j) (ω = 1, 2, ..., F, j = 1, 2, ..., J) based on the signal picked up by the spherical microphone array, and the additional microphone Frequency-domain sound pickup signal u _c (ω, j _c ) (ω = 1, 2,…, F, j = 1, 2,…, J _c ) based on signals collected by the array group and sound field estimation Virtual microphone frequency domain sound pickup signal u (ω, r) (ω = 1) that is a sound pickup signal in the frequency domain at the virtual microphone position r , 2, ..., F) for estimating the sound field,
The frequency domain collected signal u _s (ω, j) (ω = 1, 2, ..., F, j = 1, 2, ..., J) and the frequency domain collected signal u _c (ω, j _c ) ( ω = 1, 2, ..., F, j = 1, 2, ..., J _c ), using the cost function J (ω) (ω = 1, 2, ..., F) Sound field expression vector a (ω) (ω = 1), which is a vector composed of the decomposition factor a _p (ω, q) (ω = 1, 2,…, F, q = 1, 2,…, Q) of the mixed wave , 2, ..., F), a sound field expression vector estimator,
From the sound field expression vector a (ω) (ω = 1, 2,..., F) and the virtual microphone position r, the virtual microphone frequency domain collected signal u (ω, r) (ω = 1, 2,. , F), and a sound field calculation unit.

  The sound field can be accurately estimated in a wider range.

The figure which shows an example of a structure of the sound field estimation apparatus 100. FIG. The figure which shows an example of operation | movement of the sound field estimation apparatus 100. The figure which shows an example of a structure of an additional microphone group. The figure which shows an example of a structure of the sound field estimation apparatus 101. FIG. The figure which shows an example of a structure of the sound field estimation apparatus 200. The figure which shows an example of operation | movement of the sound field estimation apparatus 200. The figure which shows an example of a structure of the sound field estimation apparatus 201. FIG.

  Hereinafter, embodiments of the present invention will be described in detail. In addition, the same number is attached | subjected to the structure part which has the same function, and duplication description is abbreviate | omitted.

<First embodiment>
First, the open spherical spherical microphone array model will be described.

[Open ball type spherical microphone array model]
An open spherical spherical microphone array is a spherical microphone array in which each microphone is supported by a thin skeleton, but can be considered acoustically equivalent to a floating state in a hollow state. Hereinafter, it is assumed that the spherical microphone array is an open spherical spherical microphone array. In the spherical microphone array, J (J ≧ 1) microphones are arranged on a spherical surface with _a radius r _a (> 0), and the microphones are arranged at a set of elevation and azimuth angles (Θ _j , Φ _j ) It is specified by (j = 1, 2, ..., J). The center position of the spherical microphone array is the same as the origin.

The three-dimensional position r (j) of the j-th microphone (j = 1, 2, ..., J) on the spherical microphone array is the set of elevation and azimuth (Θ _j , Φ _j ) is the center position of each spherical microphone array Therefore, it is expressed by the formula (1).

Similarly, the three-dimensional position r _c (j _c ) of the j _c microphone (j _c = 1, 2,..., J _c ) in the additional microphone group is the distance, elevation angle, and azimuth measured from the center position of the spherical microphone array. Using a set of angles (r _{j_c} , Θ _{j_c} , Φ _{j_c} ), it is expressed by equation (2).

[Plane wave model]
A plane wave model is a model of a plane wave that is incident on the origin of a three-dimensional coordinate system. Specifically, a plane wave that is incident on the origin at an incident angle (θ _q , φ _q ) (q = 1, 2 , ..., Q). θ _q represents an elevation angle, φ _q represents an azimuth angle, and Q is an integer of 1 or more. Note that the Q incident angles (θ _q , φ _q ) are set, for example, so that Q plane waves can be obtained uniformly from all directions. For example, Q incident angles (θ _q , φ _q ) are set so that a plane wave and a spherical wave are incident from the direction of the apex of the regular polyhedron.

  Hereinafter, the sound field estimation apparatus 100 will be described with reference to FIGS. 1 to 3.

  As shown in FIG. 1, the sound field estimation apparatus 100 includes, for example, a short-time Fourier transform unit 110, a sound field expression vector estimation unit 120, a sound field calculation unit 130, a short-time inverse Fourier transform unit 140, and a recording unit 190. Yes. The recording unit 190 is a component that appropriately records information necessary for processing by the sound field estimation apparatus 100.

  The spherical microphone array 901 and the additional microphone group 902 are connected to the sound field estimation apparatus 100. Sound pickup signals from the spherical microphone array 901 and the additional microphone group 902 are input to the sound field estimation apparatus 100. FIG. 3 shows an example of the arrangement of additional microphone groups. The position of the additional microphone is arbitrary. For example, the additional microphones are arranged symmetrically around the spherical microphone array. In the example of FIG. 3, the additional microphones are arranged in a straight line.

The time domain sound pickup signal, which is the sound pickup signal in the time domain by the j-th microphone on the spherical microphone array, is represented as y _s (t, j). Let t be a parameter representing time, and j = 1, 2,. In addition, a time domain collected signal that is a collected signal in the time domain by the j _c microphone in the additional microphone group is represented as y _c (t, j _c ). Let t be a parameter representing time, and let j _c = 1, 2,..., J _c .

The sound field estimation apparatus 100 includes a time domain sound collection signal y _s (t, j) (j = 1, 2,..., J) collected by the spherical microphone array and a time domain sound collection collected by the additional microphone group. The virtual microphone position from the signal y _c (t, 2, j _c ) (j _c = 1, 2,..., J _c ) and the virtual microphone position r that is the position of the virtual microphone that is the target of sound field estimation. A virtual microphone time domain collected signal y (t, r) that is a collected signal in the time domain at r is estimated and output.

  Hereinafter, the operation of the sound field estimation apparatus 100 will be described with reference to FIG.

The short-time Fourier transform unit 110 uses the short-time Fourier transform to convert the time-domain sound pickup signal y _s (t, j) from the j-th microphone on the spherical microphone array into the time-domain sound pickup signal y _s (t, j ) Is converted into a frequency domain sound pickup signal u _s (n, ω, j), which is a frequency domain representation of (), and is output (step S110). , N is an index representing a frame number, ω is an index representing a frequency, and ω = 1, 2,. ) And output. In addition, the short-time Fourier transform unit 110 converts the time domain sound pickup signal y _c (t, j _c ) from the j _c microphone in the additional microphone group into the frequency of the time domain sound pickup signal y _c (t, j _c ). It is converted into a frequency domain collected signal u _c (n, ω, j _c ) that is a domain representation and output (step S110). n is an index representing a frame number, ω is an index representing a frequency, and ω = 1, 2,. ) And output.

The subsequent processing is specifically performed by the sound field expression vector estimation unit 120 and the sound field calculation unit 130 for each frame n, but in order to simplify the description, the frame number n is omitted, u _s (n, ω, j) and u _c (n, ω, j _c ) are simply expressed as u _s (ω, j) and u _c (ω, j _c ).

Further, using the frequency domain collected signal u _s (ω, j) (j = 1, 2,..., J), the frequency domain collected signal vector u _s (ω) of the spherical microphone array, which is a J-dimensional vector, is used. Is defined by equation (3) (ω = 1, 2,..., F). Further, using the frequency domain sound pickup signal u _c (ω, j) (j _c = 1, 2,..., J _c ), the frequency domain sound pickup signal vector u _m ( ω) is defined by equation (4) (ω = 1, 2,..., F).

  Note that the short-time Fourier transform unit 110 may use a method other than the short-time Fourier transform as long as it converts the time-domain signal into the frequency-domain signal.

The sound field expression vector estimation unit 120 includes the frequency domain collected signal u _s (ω, j) and u _c (ω, j _c ) (ω = 1, 2,..., F, j = 1, 2,. J, j _c = 1, 2, ..., J _c ) and decomposition coefficients a _p (ω, q) (ω = 1, 2,…, F, q = 1, 2, ..., A sound field expression vector a (ω) (ω = 1, 2,..., F) that is a vector consisting of Q) is estimated and output (step S120). Hereinafter, a method for calculating the sound field expression vector a (ω) will be described.

First, a model of a signal observed by each microphone, that is, a linear combination model of the frequency domain collected signals u _s (ω, j) and u _c (ω, j _c ) will be described.

[Signal model observed by each microphone]
Assume that the signal observed at each microphone is a collection of Q plane waves. Specifically, the frequency domain sound pickup signals u _s (ω, j) and u _c (ω, j) are approximated as linear combinations of Q plane wave frequency domain signals.

Plane wave frequency domain observation signal v _s (ω, q, j), which is the observation signal in the frequency domain of the plane wave observed by the j-th microphone (j = 1, 2, ..., J) on the spherical microphone array, in other words The signal in the frequency domain observed by the microphone when the plane wave arrives is expressed by Equation (5). ω = 1, 2, ..., F, and q = 1, 2, ..., Q.

k is the wave number determined based on the frequency index ω, and ^ k _q is a vector represented by Equation (6). Note that i is an imaginary unit, and · is an inner product symbol.

In addition, the plane wave frequency domain observation signal v _c (ω, q, j, which is an observation signal in the frequency domain of the plane wave observed by the j _c microphone (j _c = 1, 2,..., J _c ) of the additional microphone group. ) Is expressed by equation (7). ω = 1, 2, ..., F, and q = 1, 2, ..., Q.

Here, the sound field expression vector a (ω) which is a Q-dimensional vector in which the decomposition coefficients a _p (ω, 1), ..., a _p (ω, Q) of the Q plane waves constituting the sound field are arranged in order. If (ω = 1, 2, ..., F) is defined by equation (8),

The frequency domain collected signal u (ω, m, j) is composed of Q plane wave frequency domain observation signals v _s (ω, q, m, j) and Q spherical waves as shown in equations (9) and (10). It is approximated as a linear combination with the frequency domain observation signal v _c (ω, q, m, j).

  Next, a method for calculating the sound field expression vector a (ω) (ω = 1, 2,..., F) will be described.

[Calculation method of sound field expression vector a (ω)]
First, a plane wave vector d _s (ω, q) (ω = 1, 2, ..., F, q = 1, 2, ..., Q) of a spherical microphone array, which is a J-dimensional vector, is defined by Equation (11). .

Similarly, a plane wave vector d _c (ω, q) (ω = 1, 2, ..., F, q = 1, 2, ..., Q) of the additional microphone group, which is a J _c- dimensional vector, is expressed by Equation (12). Define.

Furthermore, using d _p (ω, q) and d _c (ω, aq) (ω = 1, 2,…, F, q = 1, 2,…, Q), (J + J _c ) × Q The dictionary matrix D (ω) (ω = 1, 2,..., F) is defined by equation (13).

That is, the q-th column (q = 1, 2,..., Q) of the dictionary matrix D (ω) is a plane wave with an amplitude of 1 from the direction specified by the pair of elevation angle and azimuth angle (θ _q , φ _q ) to the origin. Is composed of vectors d _p (ω, q) and d _c (ω, q) consisting of a plane microphone frequency domain observation signal of a spherical microphone array and an additional microphone group when the phase at the origin is zero. The

Dictionary matrix D (ω) and sound field expression vector a (ω), frequency domain collected signal vector u _s (ω) of the spherical microphone array, frequency domain collected signal vector u _c (ω) of the additional microphone group, Is used to define the cost function J (ω) by equation (14). ω = 1, 2, ..., F.

The first term of Equation (14) is derived from the error of linear combination of Equations (9) and (10). Further, the second term of the equation (14) is a regularization term, and is expressed using Σ | a _p (ω, q) | corresponding to the L1 norm of the sound field expression vector and the regularization parameter λ.

The sound field expression vector a (ω) is calculated as a vector that realizes the minimum value of the cost function J (ω). By using the second term of equation (14) as a regularization term, the effect of bringing the value of a _p (ω, q) closer to 0 for q other than the set of a _p (ω, q) for a specific q have. That is, a sparse sound field expression vector a (ω) with respect to q is calculated. This makes it possible to extract plane waves well by setting λ appropriately even if the number Q of plane waves used for approximation by linear combination is significantly larger than the number of microphones. It becomes possible.

  The sound field calculation unit 130 uses the sound field expression vector a (ω) estimated in step S120 and the virtual microphone position r to obtain a sound collected signal in the frequency domain at the virtual microphone position r using Expression (15). The virtual microphone frequency domain sound pickup signal u (ω, r) (ω = 1, 2,..., F) is calculated and output (step S130).

  The input virtual microphone position r can be freely set as the position of the sound field desired by the user of the sound field estimation apparatus 100.

  The short-time inverse Fourier transform unit 140 includes a virtual microphone frequency domain sound collection signal u (ω, r) (ω = 1, 2,... From (n, ω, r) (ω = 1, 2, ..., F), using the short-time inverse Fourier transform, the virtual microphone frequency domain collected signal u (ω, r) in the time domain by the virtual microphone It converts into the virtual microphone time domain sound collection signal y (t, r) which is a sound collection signal, and outputs it (step S140).

  When a method other than the short-time Fourier transform is used as a method for converting the time domain signal into the frequency domain signal, an inverse transform corresponding to the method may be used.

[Modification 1]
In the sound field estimation apparatus 100, the sound collected signals by the spherical microphone array 901 and the additional microphone group 902 are time domain sound collected signals y _s (t, j) (j = 1, 2,..., J), y _c (t, j _c ) (j _c = 1, 2,..., J _c ).

However, the sound collected signals by the spherical microphone array 901 and the additional microphone group 902 are frequency domain sound collected signals u _s (ω, j), u _c (ω, j) (ω = 1, 2,..., F, j = 1 , 2, ..., J, j _c = 1, 2, ..., J _c ).

FIG. 4 shows a sound field estimation apparatus 101 configured to receive the frequency domain sound pickup signals u _s (ω, j) and u _c (ω, j). As illustrated in FIG. 4, the sound field estimation apparatus 101 includes, for example, a sound field expression vector estimation unit 120, a sound field calculation unit 130, and a recording unit 190. The sound field expression vector estimation unit 120, the sound field calculation unit 130, and the recording unit 190 are the same as the sound field expression vector estimation unit 120, the sound field calculation unit 130, and the recording unit 190 of the sound field estimation apparatus 100, respectively.

The sound field estimation apparatus 101 includes a frequency domain collected signal u _s (ω, 1, j) collected by the spherical microphone array and a frequency domain collected signal u _c (ω, j) (collected by the additional microphone group). ω = 1, 2, ..., F, j = 1, 2, ..., J, j _c = 1, 2, ..., J _c ), and a virtual microphone that is the location of the virtual microphone that is the target of sound field estimation From the position r, a virtual microphone frequency domain collected signal u (ω, r), which is a collected signal in the frequency domain at the virtual microphone position r, is estimated and output.

  According to this embodiment, the sound field is regarded as a collection of plane waves, and the sound field of the additional microphone group is collected together with the sound collection signal of the spherical microphone array when estimating the sound field by obtaining the decomposition coefficient in the frequency domain. By using it, it becomes possible to widen the spatial range in which the sound field can be estimated with high accuracy. In particular, a sparse sound field expression vector is calculated and approximated by using a cost function J (ω) defined using a regularization term including a term corresponding to the L1 norm of the sound field expression vector composed of decomposition coefficients. It is possible to successfully extract plane waves to be used for the operation.

<Second embodiment>
In the first embodiment, the method of estimating the sound field from the collected sound signal collected using the open spherical spherical microphone array has been described. In the present embodiment, a method for estimating a sound field from a collected sound signal collected using a hard sphere type spherical microphone array will be described.

  First, the hard sphere type spherical microphone array model will be described.

[Hard sphere type spherical microphone array model]
The spherical microphone array is assumed to be a hard sphere type spherical microphone array. In the spherical microphone array, J (J ≧ 1) microphones are arranged on a spherical surface with _a radius r _a (> 0), and the microphones are arranged at a set of elevation and azimuth angles (Θ _j , Φ _j ) It is specified by (j = 1, 2, ..., J). The center position of the spherical microphone array is the same as the origin.

  That is, the hard sphere type spherical microphone array model is the same as the spherical microphone array model of the first embodiment except that the spherical microphone array is changed to a hard sphere type spherical microphone array.

[Plane wave model]
The plane wave model is the same as that of the first embodiment.

  Hereinafter, the sound field estimation apparatus 200 will be described with reference to FIGS. As shown in FIG. 4, the sound field estimation apparatus 200 includes, for example, a short-time Fourier transform unit 110, a sound field expression vector estimation unit 220, a sound field calculation unit 230, a short-time inverse Fourier transform unit 140, and a recording unit 190. Yes. The recording unit 190 is a component that appropriately records information necessary for processing of the sound field estimation apparatus 200.

  The spherical microphone array 903 and the additional microphone group 904 are connected to the sound field estimation apparatus 200. The collected sound signals from the spherical microphone array 903 and the additional microphone group 904 are input to the sound field estimation apparatus 200.

The sound field estimation apparatus 200 includes a time domain sound signal y _s (t, 1, j) (j = 1, 2,..., J) collected by the spherical microphone array and a time domain collected by the additional microphone group. From the collected sound signal y _c (t, 2, j) (j _c = 1, 2,..., J _c ) and the virtual microphone position r that is the position of the virtual microphone that is the target of sound field estimation, the virtual microphone A virtual microphone time domain collected signal y (t, r), which is a collected signal in the time domain at the position r, is estimated and output.

  The operation of the sound field estimation apparatus 200 will be described with reference to FIG.

The short-time Fourier transform unit 110 performs time-domain sound pickup signals y _s (t, j) and y _c (t, j _c ) (j = 1, 2,..., J, j _c = 1, 2,. _c ) using a short-time Fourier transform, the frequency domain sound collected signal u _s (n, ω), which is the frequency domain representation of the time domain sound collected signal y _s (t, j), y _c (t, j _c ) , j) and u _c (n, ω, j _c ) and output (step S110). n is an index representing a frame number, ω is an index representing a frequency, and ω = 1, 2,.

The subsequent processing, specifically, the processing in the sound field expression vector estimation unit 220 and the sound field calculation unit 230 is executed for each frame n, as in the first embodiment, and the description is simplified. Therefore, the frame number n is omitted and u _s (n, ω, j) is simply expressed as u _s (ω, j).

The sound field expression vector estimation unit 220 performs frequency domain collected signals u _s (ω, j), u _c (ω, j _c ) (ω = 1, 2,..., F, j = 1, 2,. , J _c = 1, 2, ..., J _c ), the decomposition coefficient a _p (ω, q) (ω = 1, 2,…, F, q = 1, 2,…, Q) A sound field expression vector a (ω) (ω = 1, 2,..., F) that is a vector consisting of is estimated and output (step S220). Hereinafter, a method for calculating the sound field expression vector a (ω) will be described.

  First, a model of a signal observed by each microphone will be described.

[Signal model observed by each microphone]
As in the first embodiment, the frequency domain sound pickup signals u _s (ω, j) and u _c (ω, j _c ) are approximated as linear combinations of frequency plane signals of Q plane waves.

First, the plane wave frequency domain observation signal v _s (ω, q, j), which is the observation signal in the frequency domain of the plane wave observed by the jth microphone (j = 1, 2, ..., J) on the spherical microphone array explain. ω = 1, 2, ..., F, and q = 1, 2, ..., Q.

Assume a hard spherical spherical microphone array having the same characteristics as the spherical microphone array 903, and consider the case where the center of the hard spherical spherical microphone array coincides with the origin of the three-dimensional coordinate system. At this time, if a plane wave with an amplitude of 1 is incident from the direction specified by the pair of elevation angle and azimuth angle (θ _q , φ _q ), considering that the sound field is composed of incident waves and scattered waves, a hard sphere type The observation signal v _s (ω, q, j) in the frequency domain of the plane wave observed by the j-th microphone (j = 1, 2,..., J) on the spherical microphone array is expressed by the equation (16 ). ω = 1, 2, ..., F, and q = 1, 2, ..., Q.

Y _n ^w (x, y) is a spherical harmonic function of order n, w, Y _n ^{w *} (x, y) is a complex conjugate of a spherical harmonic function of order n, w, and j _n (x) is It is a spherical Bessel function of order n, and h _n (x) is a first kind Hankel function of order n. J _n ′ (x) and h _n ′ (x) are differential functions of j _n (x) and h _n (x), respectively. Note that k is the wave number determined based on the frequency index ω.

Next, the plane wave frequency domain observation signal v _c (ω, q, which is an observation signal in the frequency domain of the plane wave observed by the j _c microphone (j _c = 1, 2,..., J _c ) in the additional microphone group. , j _c ) will be described. ω = 1, 2, ..., F, and q = 1, 2, ..., Q.

As before, a rigid spherical spherical microphone array having the same characteristics as the spherical microphone array 903 is assumed, and the center of the rigid spherical spherical microphone array coincides with the origin of the three-dimensional coordinate system. At this time, the observation signal v _c (ω, q, j _c ) in the frequency domain of the plane wave observed by the j _c microphone (j _c = 1, 2, ..., J _c ) in the additional microphone group is non- From Patent Document 2, it is expressed by the equation (17) (ω = 1, 2,..., F, q = 1, 2,..., Q).

The frequency domain collected signal u (ω, m, j) is obtained by plane wave frequency domain observation signal v _s (ω, q, j) in equation (16) and plane wave frequency domain observation signal v _c (ω, q, m, j _c ) is used as an approximate representation as the same linear combination as equations (9) and (10).

[Calculation method of sound field expression vector a (ω)]
Using the plane wave frequency domain observation signal v _s (ω, q, j) defined by Equation (16) and the plane wave frequency domain observation signal v _c (ω, q, j _c ) defined by Equation (17), The dictionary matrix D (ω) is defined similarly to equation (13), and the cost function J (ω) is defined similarly to equation (14). The sound field expression vector a (ω) can be calculated to result in solving the Lasso-type optimization problem similar to the first embodiment.

Note that the observed signal v _s (ω, q, j) in equation (16) and the observed signal v _c (ω, q, j _c ) in equation (17) are defined with a limit (infinite sum of n). Therefore, in practice, the observed signal v _s (ω, q, j) and the observed signal v _c (ω are calculated by numerical calculation using finite n (hereinafter, N is an integer greater than or equal to 0). , q, j). In other words, instead of Equation (16) and Equation (17), Equation (16) ′ and Equation are used as the calculation formula for observation signal v _s (ω, q, j) and observation signal v _c (ω, q, j). (17) 'is used.

For example, when r _a = 4 cm, it take approximately N = 10.

  The sound field calculation unit 230 uses the sound field expression vector a (ω) estimated in step S220 and the virtual microphone position r to obtain a sound collected signal in the frequency domain at the virtual microphone position r using Equation (18). A certain virtual microphone frequency domain collected signal u (ω, r) (ω = 1, 2,..., F) is calculated and output (step S230).

Here, (Θ ⁺ , Φ ⁺ ) is a set of the elevation angle and azimuth angle of the virtual microphone position r viewed from the origin of the three-dimensional coordinate system.

  The short-time inverse Fourier transform unit 140 includes a virtual microphone frequency domain sound collection signal u (ω, r) (ω = 1, 2,... From (n, ω, r) (ω = 1, 2, ..., F), the short-time inverse Fourier transform is used to convert the virtual microphone frequency domain collected signal u (ω, r) in the time domain by the virtual microphone. It converts into the virtual microphone time domain sound collection signal y (t, r) which is a sound collection signal, and outputs it (step S140).

[Modification 2]
Similarly to the first modification of the first embodiment, the sound field estimation apparatus 200 may be configured. That is, in the sound field estimation apparatus 200, the sound collected signals by the spherical microphone array 903 and the additional microphone group 904 are time domain sound collected signals y _s (t, j), y _c (t, j _c ) (j = 1, 2 ,..., J, j _c = 1, 2,..., J _c ), but the collected sound signals by the spherical microphone array 903 and the additional microphone group 904 are the frequency domain collected signals u _s (ω, j). , U _c (ω, j _c ) (ω = 1, 2,..., F, j = 1, 2,..., J, j _c = 1, 2,..., J _c ).

FIG. 7 shows a sound field estimation apparatus 201 that receives the frequency domain collected signals u _s (ω, j) and u _c (ω, j _c ) as inputs. As shown in FIG. 7, the sound field estimation apparatus 201 includes a sound field expression vector estimation unit 220, a sound field calculation unit 230, and a recording unit 190. The sound field expression vector estimation unit 220, the sound field calculation unit 230, and the recording unit 190 are the same as the sound field expression vector estimation unit 220, the sound field calculation unit 230, and the recording unit 190 of the sound field estimation apparatus 200, respectively.

The sound field estimation apparatus 201 includes a frequency domain collected signal u _s (ω, j) collected by the spherical microphone array and a frequency domain collected signal u _c (ω, j _c ) (ω = 1, 2, ..., F, j = 1, 2, ..., J, j _c = 1, 2, ..., J _c ) and the virtual microphone position that is the placement position of the virtual microphone that is the target of sound field estimation From r, a virtual microphone frequency domain collected signal u (ω, r) that is a collected signal in the frequency domain at the virtual microphone position r is estimated and output.

  According to the invention of this embodiment, the sound field is regarded as a collection of plane waves, and when the sound field is estimated by obtaining the decomposition coefficient in the frequency domain, the sound from the additional microphone group is collected together with the collected sound signal from the spherical microphone array. By using the collected sound signal, it is possible to expand the spatial range in which the sound field can be estimated with high accuracy. In particular, a sparse sound field expression vector is calculated and approximated by using a cost function J (ω) defined using a regularization term including a term corresponding to the L1 norm of the sound field expression vector composed of decomposition coefficients. It is possible to successfully extract plane waves to be used for the operation.

<Supplementary note>
The apparatus of the present invention includes, for example, a single hardware entity as an input unit to which a keyboard or the like can be connected, an output unit to which a liquid crystal display or the like can be connected, and a communication device (for example, a communication cable) capable of communicating outside the hardware entity. Can be connected to a communication unit, a CPU (Central Processing Unit, may include a cache memory or a register), a RAM or ROM that is a memory, an external storage device that is a hard disk, and an input unit, an output unit, or a communication unit thereof , A CPU, a RAM, a ROM, and a bus connected so that data can be exchanged between the external storage devices. Further, if necessary, a hardware entity may be provided with a device (drive) that can read and write a recording medium such as a CD-ROM. A physical entity having such hardware resources includes a general-purpose computer.

  The external storage device of the hardware entity stores a program necessary for realizing the above functions and data necessary for processing the program (not limited to the external storage device, for example, reading a program) It may be stored in a ROM that is a dedicated storage device). Data obtained by the processing of these programs is appropriately stored in a RAM or an external storage device.

  In the hardware entity, each program stored in an external storage device (or ROM or the like) and data necessary for processing each program are read into a memory as necessary, and are interpreted and executed by a CPU as appropriate. . As a result, the CPU realizes a predetermined function (respective component requirements expressed as the above-described unit, unit, etc.).

  The present invention is not limited to the above-described embodiment, and can be appropriately changed without departing from the spirit of the present invention. In addition, the processing described in the above embodiment may be executed not only in time series according to the order of description but also in parallel or individually as required by the processing capability of the apparatus that executes the processing. .

  As described above, when the processing functions in the hardware entity (the apparatus of the present invention) described in the above embodiments are realized by a computer, the processing contents of the functions that the hardware entity should have are described by a program. Then, by executing this program on a computer, the processing functions in the hardware entity are realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like, and as an optical disk, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only). Memory), CD-R (Recordable) / RW (ReWritable), etc., magneto-optical recording medium, MO (Magneto-Optical disc), etc., semiconductor memory, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. Can be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. When executing the process, the computer reads a program stored in its own recording medium and executes a process according to the read program. As another execution form of the program, the computer may directly read the program from a portable recording medium and execute processing according to the program, and the program is transferred from the server computer to the computer. Each time, the processing according to the received program may be executed sequentially. Also, the program is not transferred from the server computer to the computer, and the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by the execution instruction and result acquisition. It is good. Note that the program in this embodiment includes information that is used for processing by an electronic computer and that conforms to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer).

  In this embodiment, a hardware entity is configured by executing a predetermined program on a computer. However, at least a part of these processing contents may be realized by hardware.

Claims (4)

J is the number of microphones on the spherical microphone array, and (Θ _j , Φ _j ) (j = 1, 2,..., J) is a set of elevation and azimuth angles that specify the placement position of the microphone. the radius of the array and r _a, the center position of the spherical microphone array as the origin, a number of additional microphones group microphone and J _c, the incident angle to the origin of the three-dimensional coordinate system plane wave model (θ _q, φ _q) In the frequency domain of the plane wave observed by the jth microphone (j = 1, 2, ..., J) on the spherical microphone array. Plane wave frequency domain observation signal of v _s (ω, q, j) (ω is an index representing frequency, and ω = 1, 2, ..., F, q = 1, 2, ..., Q ), and the j _c microphone in the additional microphone group _{(j c = 1, 2,} ..., the observed signal der in the frequency domain of a plane wave observed in J _c) The plane wave frequency domain observed signal _{v c (ω, q, j} ) (ω = 1, 2, ..., F, q = 1, 2, ..., Q) _{and, d s (ω, q)} (ω = 1, 2, ..., F, q = 1, 2, ..., Q) as the J-dimensional q-plane wave vector defined by the following equation:

Let d _c (ω, q) (ω = 1, 2, ..., F, q = 1, 2, ..., Q) be the J _c- dimensional q-th spherical wave vector defined by the following equation:

Let D (ω) (ω = 1, 2, ..., F) be a (J + J _c ) × Q dictionary matrix defined by the following equation:

Using λ as a regularization parameter, the cost function J (ω) (ω = 1, 2, ..., F) is expressed as the sound field expression vector a (ω) (ω = 1, 2, ..., F) and the decomposition coefficient a _p J (ω) defined by the following equation using (ω, q) (ω = 1, 2, ..., F, q = 1, 2, ..., Q)

The sound field expression vector a (ω) (ω = 1, 2,..., F) is the decomposition coefficient a _p (ω, q) (ω = 1, 2,... F, q = 1, 2,. , Q) to be a Q-dimensional vector defined by

u _s (ω), u _c (ω) (ω = 1, 2,…, F) are added to the frequency domain sound signal vector and J _c dimension of the J-dimensional spherical microphone array defined by the following equations, respectively. As the frequency domain sound pickup signal vector of the microphone group,

Frequency domain sound pickup signal u _s (ω, j) (ω = 1, 2, ..., F, j = 1, 2, ..., J) based on the signal picked up by the spherical microphone array, and the additional microphone Frequency-domain sound pickup signal u _c (ω, j _c ) (ω = 1, 2,…, F, j = 1, 2,…, J _c ) based on signals collected by the array group and sound field estimation Virtual microphone frequency domain sound pickup signal u (ω, r) (ω = 1) that is a sound pickup signal in the frequency domain at the virtual microphone position r , 2, ..., F) for estimating the sound field,
The frequency domain collected signal u _s (ω, j) (ω = 1, 2, ..., F, j = 1, 2, ..., J) and the frequency domain collected signal u _c (ω, j _c ) ( ω = 1, 2, ..., F, j _c = 1, 2, ..., J _c ), using the cost function J (ω) (ω = 1, 2, ..., F) Sound field expression vector a (ω) (ω =), which is a vector composed of decomposition coefficients a _p (ω, q) (ω = 1, 2,…, F, q = 1, 2,…, Q) 1, 2, ..., F), a sound field expression vector estimator,
From the sound field expression vector a (ω) (ω = 1, 2,..., F) and the virtual microphone position r, the virtual microphone frequency domain collected signal u (ω, r) (ω = 1, 2,. , F)
A sound field estimation apparatus including: The sound field estimation apparatus according to claim 1,
The time domain sound pickup signal y _s (t, j) by the j-th microphone on the spherical microphone array is changed to the frequency domain sound pickup signal u _s (ω, j) (ω = 1, 2,..., F, j = 1 , 2,..., J) and the time domain collected signal y _c (t, j _c ) by the j _c microphone in the additional microphone group is converted to the frequency domain collected signal u _c (ω, j _c). ) a short-time Fourier transform unit for converting to (ω = 1, 2, ..., F, j _c = 1, 2, ..., J _c ),
Convert the above virtual microphone frequency domain collected signal u (ω, r) (ω = 1, 2, ..., F) into time domain collected signal y (t, r) (ω = 1, 2, ..., F) A short-time inverse Fourier transform unit,
A sound field estimation apparatus further comprising: J is the number of microphones on the spherical microphone array, and (Θ _j , Φ _j ) (j = 1, 2,..., J) is a set of elevation and azimuth angles that specify the placement position of the microphone. the radius of the array and r _a, the center position of the spherical microphone array as the origin, a number of additional microphones group microphone and J _c, the incident angle to the origin of the three-dimensional coordinate system plane wave model (θ _q, φ _q) In the frequency domain of the plane wave observed by the jth microphone (j = 1, 2, ..., J) on the spherical microphone array. Plane wave frequency domain observation signal of v _s (ω, q, j) (ω is an index representing frequency, and ω = 1, 2, ..., F, q = 1, 2, ..., Q ), and the j _c microphone in the additional microphone group _{(j c = 1, 2,} ..., the observed signal der in the frequency domain of a plane wave observed in J _c) The plane wave frequency domain observed signal _{v c (ω, q, j} ) (ω = 1, 2, ..., F, q = 1, 2, ..., Q) _{and, d s (ω, q)} (ω = 1, 2, ..., F, q = 1, 2, ..., Q) as the J-dimensional q-plane wave vector defined by the following equation:

Let d _c (ω, q) (ω = 1, 2, ..., F, q = 1, 2, ..., Q) be the J _c- dimensional q-th spherical wave vector defined by the following equation:

Let D (ω) (ω = 1, 2, ..., F) be a (J + J _c ) × Q dictionary matrix defined by the following equation:

Using λ as a regularization parameter, the cost function J (ω) (ω = 1, 2, ..., F) is expressed as the sound field expression vector a (ω) (ω = 1, 2, ..., F) and the decomposition coefficient a _p J (ω) defined by the following equation using (ω, q) (ω = 1, 2, ..., F, q = 1, 2, ..., Q)

The sound field expression vector a (ω) (ω = 1, 2,..., F) is the decomposition coefficient a _p (ω, q) (ω = 1, 2,... F, q = 1, 2,. , Q) to be a Q-dimensional vector defined by

u _s (ω), u _c (ω) (ω = 1, 2,…, F) are added to the frequency domain sound signal vector and J _c dimension of the J-dimensional spherical microphone array defined by the following equations, respectively. As the frequency domain sound pickup signal vector of the microphone group,

Frequency domain sound pickup signal u _s (ω, j) (ω = 1, 2, ..., F, j = 1, 2, ..., J) based on the signal picked up by the spherical microphone array, and the additional microphone Frequency-domain sound pickup signal u _c (ω, j _c ) (ω = 1, 2,…, F, j = 1, 2,…, J _c ) based on signals collected by the array group and sound field estimation Virtual microphone frequency domain sound pickup signal u (ω, r) (ω = 1) that is a sound pickup signal in the frequency domain at the virtual microphone position r , 2, ..., F) for estimating the sound field,
The sound field expression vector estimation unit is configured to perform the frequency domain sound collection signal u _s (ω, j) (ω = 1, 2,..., F, j = 1, 2,. _c (ω, j _c ) (ω = 1, 2, ..., F, j _c = 1, 2, ..., J _c ), the cost function J (ω) (ω = 1, 2, ..., F) Is used to express the sound field that is a vector consisting of the decomposition coefficients a _p (ω, q) (ω = 1, 2,…, F, q = 1, 2,…, Q) of the mixed wave that composes the sound field. A sound field expression vector estimation step for estimating a vector a (ω) (ω = 1, 2,..., F);
From the sound field expression vector a (ω) (ω = 1, 2,..., F) and the virtual microphone position r, the sound field calculation unit calculates the virtual microphone frequency domain collected signal u (ω, r) (ω = 1, 2, ..., F) calculating the sound field;
Sound field estimation method including   The program for functioning a computer as each part of the sound field estimation apparatus of Claim 1 or 2.

Images