JP2017130899A - Sound field estimation device, method therefor and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a sound field estimation device having a larger space region in which sound field estimation is effective than prior arts, a method therefor and a program.SOLUTION: A sound field estimation device 200 includes: a plane wave decomposition unit 213 for estimating a vector consisting of a strength of a plane wave constituting a sound field using a sound collection signal u(ω, m, j) in frequency domain of M spherical microphone arrays 1 and 2 including microphones at predetermined positions on a sphere; and an interpolation estimation unit 216 for estimating a sound collection signal u^(ω, r) in frequency domain at a position rusing an estimate a(ω) of the vector consisting of the strength of the plane wave and the position rof the virtual microphone.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a technique for estimating a sound collection signal obtained when a microphone is disposed at another position using a sound collection signal of a microphone disposed at a certain position.

  In recent years, audio playback technology has expanded from 2-channel stereo to 5.1-channel playback, and further research and development on 22.2 channel playback and wavefront synthesis methods have greatly improved the realism of reproduction itself. It is intended to expand the high playback area as much as possible.

  In order to evaluate and verify such a multi-channel audio reproduction method, it is important to measure the reproduced sound field. For example, in the wavefront synthesis method, it is necessary to compare the actually recorded sound field with the reproduced sound field and grasp the difference. The reason for this is that various factors such as signal processing for converting the recorded sound field into a playback signal, encoding and decoding of the recorded signal, and acoustic characteristics of the room where the playback device is installed affect the sound field reproduction accuracy. This is because it is important to establish a method with high reproducibility.

(Conventional method 1)
As a method for measuring the sound field, it is conceivable to concentrate microphones locally in a part of the target measurement area and estimate the sound field in the surrounding area from the measurement result. As an example, a spherical microphone array is being studied. The spherical microphone array, a microphone array configured by arranging the several tens or more microphone element on a sphere of radius r _a, the r _a is in the range from a few cm of ten cm.

FIG. 1 shows a signal flow of sound field estimation processing using a spherical microphone array 1 in the prior art. The time domain signals y (t, r _a , Ω _j ) respectively picked up by the J microphones arranged on the spherical surface are converted by the short-time Fourier transform unit 111 into the frequency domain signals u (i, ω, r). _a , Ω _j ). Where t is time, i is a frame, J is an integer of 2 or more, ω is a time frequency, and j = 1, 2,. Note that in the following processing, processing is performed in units of frames, but i is omitted to simplify the notation. Ω _j is a position on the spherical surface of the j-th microphone element, and is specified by a pair of elevation angle θ _j and azimuth angle φ _j . Ω _j = (θ _j , φ _j ).

The spherical wave spectrum conversion unit 112 obtains _a spherical wave spectrum u _{n, m} (ω, r _a ) by the following equation for each frequency ω.

However, α _j is a weight appropriately set so that the product sum of Equation (1) satisfies the orthogonal condition of the spherical harmonic function expressed by the following equation.

Y _n ^m (θ _j , φ _j ) is a spherical harmonic function of order n and order m, and * means a complex conjugate. n = 0,1, ..., N, m = -n, -n + 1, ..., n. δ _{nn ′} is 1 when n = n ′, 0 when n ≠ n ′, δ _{mm ′} is 1 when m = m ′, and 0 when m ≠ m ′. Value. To obtain a spherical wave spectrum up to the order number N, (N + 1) ² or more microphone elements are required.

  In the following, the sound field generated by the sound source outside the measurement target range is measured, that is, the internal problem is dealt with. In other words, the sound field generated by the sound source outside the sphere formed by the spherical microphone array is measured.

  The sound field is considered in the polar coordinate system (r, Ω) = (r, θ, φ) with the center of the spherical microphone array as the origin.

In extrapolation estimator 116, the following equation in the frequency omega, extrapolating the sound field to the radius r from the position of the radius r _a, the polar coordinate system (r, Ω) = (r , θ, φ) collected sound in the signal u Find (ω, r, Ω). In other words, the sound field outside the sphere formed by the spherical microphone array is estimated using the collected sound signals of the microphones arranged on the spherical microphone array.

Where k is the wave number k = ω / c (c is the speed of sound), and b _n () is the mode intensity function.

  Non-Patent Document 1 shows a case of an open sphere spherical microphone array in which microphone elements are hollow and arranged on a spherical surface. In this case, the mode intensity function is expressed by the following equation.

It is. Here, i is an imaginary number, and j _n () is an nth order spherical Bessel function. When the spherical microphone array is configured by arranging microphone elements on the surface of a hard sphere, the mode intensity function is expressed by the following expression based on Non-Patent Document 2.

Here, h _n () is an nth-order first-class Hankel function. “A ′” means the differentiation of A.

  The short-time inverse Fourier transform unit 118 returns the spatially extrapolated sound collection signal from the frequency domain signal u (ω, r, Ω) to the time domain signal y (t, r, Ω) and outputs it.

In Equation (3), b _n (kr) / b _n (kr _a ) is applied to the spherical wave spectrum, and the product sum is taken as Y _n ^m (θ, φ). This product sum of Y _n ^m (θ, φ) corresponds to inverse spherical wave spectrum conversion. Therefore, the spatially extrapolated sound pickup signal u (ω, r, Ω) is a frequency domain signal.

In the measurement using the open-ball type microphone array, the influence of the singular point cannot be avoided, and it becomes impossible to measure at k and r where j _n (kr) = 0. Specifically, when j _n (kr) = 0 is satisfied even if a sound field exists, its output becomes zero. However, the hard sphere type microphone array has no singular point and does not become impossible to measure. For this reason, as the spherical microphone array, it is mainstream to use a rigid sphere type microphone array.

T. Abhayapala and D. Ward, "Theory and design of high order sound field microphones using spherical microphone array", in Acoustics, Speech, and Signal Processing (ICASSP), IEEE International Conference on, 2002, pp. II-1949. Meyer, Jens; Elko, Gary, "A highly scalable spherical microphone array based on an orthonormal decomposition of the soundfield", Acoustics, Speech, and Signal Processing (ICASSP), 2002 IEEE International Conference on, 2002, II-1781-II- 1784.

In the prior art, when the position of the radius r _a extrapolating the sound field to the radius r, using the Bessel function j _n (kr) or Bessel functions j _n (kr) and Hankel function h _n (kr) ing. According to the reference 1, when kr increases as a global trend in both functions, it decreases at a rate of 1 / kr.
(Reference 1) G. Williams, “Fourier Acoustics”, Springer Fairlark, 2005, p.234-236.
For example, if r is 10 times r _a, the estimated value of the extrapolation becomes abruptly small as about 1/10. For this reason, the spatial region in which extrapolation is effective is limited to the periphery of the spherical microphone array surface. For the same reason, even if the frequency ω increases, k = ω / c increases, and the extrapolated estimated value decreases rapidly. In other words, when the frequency is increased, the spatial region in which extrapolation is effective is rapidly narrowed.

  An object of the present invention is to provide a sound field estimation apparatus, a method and a program therefor, which have a larger spatial region in which sound field estimation is more effective than the prior art.

In order to solve the above problems, according to one aspect of the present invention, the sound field estimation apparatus uses m = 1, 2,..., M as an index of a spherical microphone array, and j _m = 1, 2,. J _m , r _m are polar coordinate _radials , θ _{j_m} and φ _{j_m} are polar coordinate declinations, ω is a time frequency index, and declination angles θ _{j_m} and φ _{j_m} on a sphere of radius r _m centered at d ^- _m of J _m-number of position _{^{r - (m, j m)}} = d - m + [r m sinθ j_m cosφ j_m r m sinθ j_m sinφ j_m r m cosθ j_m] M pieces of each ^T comprises a microphone spherical microphone array m A plane wave decomposition unit that estimates a vector composed of the intensity of a plane wave that constitutes the sound field using a sound collection signal u (ω, m, j _m ) in the frequency domain, and an estimated value a ( ω) and the virtual microphone position r ^- _p, and an interpolation estimation unit that estimates the frequency domain sound pickup signal u ^ (ω, r ^- _p ) at the virtual microphone position r ^- _p. Including.

In order to solve the above problems, according to another aspect of the present invention, a sound field estimation method includes m = 1, 2,..., M as an index of a spherical microphone array, j _m = 1, 2,. J _m and r _m are polar coordinate _radials , θ _{j_m} and φ _{j_m} are polar coordinate declinations, ω is an index of time frequency, and the plane wave decomposition part is a deviation on a sphere with a radius r _m centered on d ^- _m. angle theta _{J_m} and phi _{J_m} of J _m-number of position _{^{r - (m, j m)}} = d - m + [r m sinθ j_m cosφ j_m r m sinθ j_m sinφ j_m r m cosθ j_m] each ^T comprises a microphone M A plane wave decomposition step for estimating a vector consisting of the intensity of plane waves constituting the sound field using the collected sound signal u (ω, m, j _m ) in the frequency domain of each spherical microphone array m, and an interpolation estimation unit, estimate of vector of intensity of a plane wave with a (ω), the position r of the virtual microphone ^- with a _p, the position of the virtual microphone r ^- collected sound signal in the frequency domain in _p u ^ (ω, r ^- and a interpolation estimation step of estimating _p).

  According to the present invention, there is an effect that a spatial region in which sound field estimation is effective is larger than that of the related art.

The functional block diagram of the sound field estimation apparatus which concerns on a prior art. The functional block diagram of the sound field estimation apparatus which concerns on 1st embodiment. The figure which shows the example of the processing flow of the sound field estimation apparatus which concerns on 1st embodiment. The figure which shows the outline | summary of the position of the virtual microphone in 1st embodiment and the modification 1. FIG. The functional block diagram of the sound field estimation apparatus which concerns on 2nd embodiment. The figure which shows the example of the processing flow of the sound field estimation apparatus which concerns on 2nd embodiment. The functional block diagram of the sound field estimation apparatus which concerns on 3rd embodiment. The figure which shows the example of the processing flow of the sound field estimation apparatus which concerns on 3rd embodiment.

Hereinafter, embodiments of the present invention will be described. In the drawings used for the following description, constituent parts having the same function and steps for performing the same process are denoted by the same reference numerals, and redundant description is omitted. In the following explanation, the symbols "^", " ^- ", etc. used in the text should be written immediately above the character just before, but they are written immediately after the character due to restrictions on the text notation. . In the formula, these symbols are written in their original positions. Further, the processing performed for each element of a vector or matrix is applied to all elements of the vector or matrix unless otherwise specified.

<Points of first embodiment>
In this embodiment, (1) a plurality of microphone arrays are used instead of a single spherical microphone array, and (2) instead of a spherical wave spectrum, a collection of plane waves that constitute a sound field from a collected signal in the frequency domain. The sound field is estimated directly using this plane wave. (1) and (2) make it possible to greatly expand the spatial range in which sound field estimation is effective. The method will be described below.

<Sound Field Estimation Device 200 according to First Embodiment>
FIG. 2 is a functional block diagram of the sound field estimation apparatus 200 according to the first embodiment, and FIG. 3 shows its processing flow.

  The sound field estimation apparatus 200 includes a short-time Fourier transform unit 211, a plane wave decomposition unit 213, an interpolation estimation unit 216, and a short-time inverse Fourier transform unit 218.

The sound field estimation apparatus 200 receives a time domain collected signal y (t, 1, j) (where j = 1, 2,..., J) from the spherical microphone array 1 and receives a time domain collected signal y from the spherical microphone array 2. Receives the collected sound signal y (t, 2, j) (where j = 1, 2,..., J), receives the virtual microphone position information r ^−, and collects the time domain sound collected signal at the virtual microphone position r ⁻ y (t, r ^-) to estimate, to output.

In this embodiment, it is assumed that the number of microphones on the spherical surfaces included in the spherical microphone arrays 1 and 2, the arrangement of the microphones in the spherical microphone arrays 1 and 2, and the radius of the spherical microphone arrays 1 and 2 are the same. The spherical microphone arrays 1 and 2 are open-type spherical microphone arrays in which J microphones are arranged on _a spherical surface with _a radius ra, and the microphone arrangement on the spherical surface is a pair of elevation angle and azimuth angle (θ _j , φ _j ). The center position of the spherical microphone array 1 d ^- ₁ = a _{[x 1, y 1, z} 1], the center position of the spherical microphone array 2 d ^- ₂ = a _{[x 2, y 2, z} 2]. The three-dimensional position of the jth microphone on the spherical microphone array m (m = 1, 2) is

Given in. The collected sound signal from the microphone is assumed to be a collected sound signal y (t, m, j) (where m = 1, 2, j = 1, 2,..., J) in the time domain.

<Short-time Fourier transform unit 211>
The short-time Fourier transform unit 211 receives a time-domain sound pickup signal y (t, m, j) (where m = 1, 2, j = 1, 2,..., J), Time-domain sound pickup signal y (t, m, j) is changed to frequency-domain sound pickup signal u (i, ω, m, j) (where i is a frame number, ω = 1, 2,..., F, j = 1, 2,..., J) (S211) and output. The subsequent processing is performed for each frame i, but the frame number i is omitted to simplify the description. Further, any method other than the short-time Fourier transform may be used as long as it is a method for converting a time-domain signal into a frequency-domain signal.

<Plane wave decomposition unit 213>
The plane wave decomposition unit 213 generates a frequency-domain sound pickup signal u (ω, m, j) (where ω = 1, 2,..., F, m = 1, 2, j = 1, 2,..., J). Using these values, a vector composed of the intensity of the plane wave constituting the sound field is estimated (S213), and an estimated value a ⁻ (ω) (where ω = 1, 2,..., F) is output. . For example, the plane wave decomposition unit 213 obtains a solution vector (estimated value) a ⁻ (ω) that minimizes the next cost function J in order to obtain a collection of plane waves constituting the sound field.

Note that λ is a constant for regularization. As λ is increased, estimation can be made more robust against noise on u (ω, m, j). Also, D (ω) in equation (7) is called a dictionary matrix,

The l-th column vector D (ω, l) is the phase at the origin when a plane wave of amplitude 1 is incident from the direction specified by the elevation angle and azimuth angle pair (θ _l , φ _l ). Is a vector composed of observation values at the spherical microphone arrays 1 and 2 in a state where is 0. For example, L ′ incident angles (θ _l , φ _l ) are set so that L ′ plane waves can be obtained uniformly from all directions. For example, L ′ incident angles (θ _l , φ _l ) are set so that plane waves are incident from the direction of the apex of the regular polyhedron.

The l-th column vector D (ω, l) of the dictionary matrix D (ω) is an observation value of the j-th microphone of the spherical microphone array m when a plane wave having an incident angle (θ _l , φ _l ) arrives.

をもちいて、

Use

で表される。

It is represented by

The l-th value of the solution vector a ⁻ (ω) corresponds to the amplitude of the l-th plane wave. a (ω) = a _{[a 1 (ω), a} 2 (ω), ..., a l (ω), ..., a L '(ω)] T.

Convex optimization of using the regularization term of L1 norm, the solution vector a ^- (ω) as derive sparse vector containing much 0. Therefore, as shown in Reference Document 2, it is possible to successfully extract plane waves even in a redundant case where the number L ′ of plane waves assumed in advance greatly exceeds the number of microphones.
(Reference 2) A. Wabnitz, N. Epain, A. van.Shaik, C. Jin, "Reconstruction of spatial sound field using compressed sensing", in Acoustics, Speech, and Signal Processing (ICASSP), IEEE International Conference on , 2011.

<Interpolation estimation unit 216>
The interpolation estimation unit 216 receives the estimated value a (ω) and the virtual microphone position information r ⁻ = (r _x , r _y , r _z ), and collects the sound signal u in the frequency domain at the position r ⁻ of the virtual microphone. ^ (ω, r ⁻ ) (where ω = 1, 2,..., F) is estimated by the following equation (S216) (in other words, from the plane wave model with the solution vector a ⁻ (ω) as a parameter, the virtual microphone the output value u ^ (ω, r ^-) to estimate), and outputs.

However, ● means an inner product, and k ^- _l is a wave vector corresponding to the incident direction of the l-th plane wave, and is expressed by the following equation.

^T represents transposition. Note that the position information r ⁻ of the virtual microphone is input by a user of the sound field estimation apparatus 200, for example.

<Short-time inverse Fourier transform unit 218>
The short-time inverse Fourier transform unit 218 receives the collected sound signal u ^ (ω, r ⁻ ) (where ω = 1, 2,..., F) in the frequency domain, and collects the collected sound signal u by inverse short-time Fourier transform. ^ (ω, r ⁻ ) is converted into a time domain sound pickup signal y (t, r ⁻ ) (S218) and output. Note that a method corresponding to the conversion method in the short-time Fourier transform unit 211 may be used as a method for converting a time-domain signal into a frequency-domain signal.

<Effect>
With the above configuration, it is possible to realize a sound field estimation apparatus having a larger spatial area in which sound field estimation is more effective than in the related art.

<Modification 1>
In the first embodiment, a single virtual microphone is assumed, and a signal collected at that position is estimated. However, of course, virtual microphones may be assumed at a plurality of positions. Further, by arranging virtual microphones on the same spherical surface, an open-ball type virtual microphone array having a radius r can be configured. For example, if the center of the virtual microphone array is at the position of D = [d _x d _y d _z ] when viewed from the origin, and a virtual microphone array including P virtual microphones is configured, the sound The field estimation apparatus 200 receives the position information (r ⁻ _p ) (where p = 1, 2,..., P) of the P virtual microphones, and collects the collected sound signal y (t, r ⁻ _p ) (where p = 1,2, ..., P) is output.

When the position of the p-th microphone on the spherical surface of the virtual microphone array is r ^- _p (where p = 1, 2,..., P), the interpolation estimation unit 216 captures the collected sound signal u ^ ( ω, r ^- _p ) is estimated by the following equation.

FIG. 4 shows an outline of the position of the virtual microphone in the first embodiment and the first modification thereof. When the center [d _x d _y d _z ] of the virtual microphone array is [r _x r _y r _z ], the radius r = 0, and the number P of virtual microphones included in the virtual microphone array is 1, Since this embodiment is an embodiment, the first embodiment can be said to be an example of Modification 1 (however, in this modification, a microphone on a spherical surface having a radius r is provided. On the other hand, in the first embodiment, a radius r = 0, that is, , A microphone is provided on the point, not on the spherical surface).

<Modification 2>
In the first embodiment, the case where two open-ball type microphone arrays are installed in the sound field has been described. In this modification, a case where two hard sphere type microphone arrays are installed in a sound field will be described.

The hard-sphere microphone array has a radius r _a and the number of microphones J, and the microphone arrangement on the spherical surface is specified by a pair of elevation angle and azimuth angle (θ _j , φ _j ). When a plane wave having an amplitude of 1 is incident from a direction specified by a pair of elevation angle and azimuth angle (θ _l , φ _l ), the sound field is composed of an incident wave and a scattered wave.

  When the center of the hard-sphere microphone array coincides with the origin of the coordinate system, the signal observed by the jth microphone is

become. If the center of the rigid ball-type microphone array of the m (m = 1,2) is away d _m from the origin, the signal observed at the j-th microphone, taking into account the phase difference

になる。

become.

  Therefore, if equation (16) is used instead of equation (9) and the l-th column vector D (ω, l) of the dictionary matrix D (ω) of equation (10) is generated, then optimization is performed similarly. By solving the problem, a plane wave can be extracted from the output signal of the hard sphere microphone array.

Since equation (15) includes an infinite number of terms, in actuality, finite n is used, and v ^rigid (ω, l, m, j) is obtained by numerical calculation. When r _a = 4 cm, n = 10 is sufficient.

<Other variations>
In the present embodiment, the microphone arrangement in the microphone arrays 1 and 2 and the radius of the spherical microphone arrays 1 and 2 are the same, but they may be different. Further, the number of microphone arrays need not be two, but may be plural. For example, M is any integer of 2 or more, and the three-dimensional position of the j _m microphone on the M spherical microphone arrays m (m = 1, 2,..., M) is the center of the spherical microphone array m. d ^- _m , radius r _m

Given in. The collected sound signal from the microphone is collected in the time domain as y (t, m, j _m ) (j _m = 1, 2,..., J _m , where J _m is the number of microphones included in the spherical microphone array m). And Also, equations (7) and (8) are replaced with the following equations.

The l-th column vector D (ω, l) of the dictionary matrix D (ω) is an observation value of the j _m microphone of the spherical microphone array m when a plane wave having an incident angle (θ _l , φ _l ) arrives.

をもちいて、

Use

で表される。

It is represented by

  In addition, formulas (15) and (16) in the second modification of the first embodiment are replaced with the following formulas.

<Second embodiment>
A description will be given centering on differences from the first modification of the first embodiment.

  In Modification 1 of the first embodiment, a sound pickup signal is estimated assuming a virtually open-ball type microphone array. In the second embodiment, based on the configuration of the first modification of the first embodiment, instead of the open-ball type microphone array, a hard-ball type microphone array is virtually assumed, and the collected sound signal is estimated.

  FIG. 5 is a functional block diagram of the sound field estimation apparatus 300 according to the second embodiment, and FIG. 6 shows the processing flow.

  The sound field estimation apparatus 300 includes a short-time Fourier transform unit 211, a plane wave decomposition unit 213, an interpolation estimation unit 216, and a short-time inverse Fourier transform unit 218, and further includes an array type conversion unit 317.

First, as a virtual spherical microphone array, sound collection by a double open sphere microphone array of Reference 3 is assumed. In this microphone array, microphone elements are arranged on a spherical surface with a radius r or a spherical surface with a radius αr, and α = 1.2 is recommended.
(Reference 3) I. Balmages, B. Rafaely, "Open-Sphere Designs for Spherical Microphone Arrays", IEEE Transactions on Audio, Speech, and Language Processing, vol. 15, no. 2, pp 727-732, 2007.
For example, it is assumed that Q = P × 2, and the positions of P virtual microphone elements among the Q virtual microphone elements are the same as those in the first modification. That is, the center of the virtual microphone array is at the position of D = [d _x d _y d _z ] when viewed from the origin, and the position of the p-th virtual microphone on the spherical surface of the virtual microphone array r ⁻ _p is

It is. Of the Q virtual microphone elements, the remaining P virtual microphones are arranged on a spherical surface with a center of [d _x d _y d _z ] and a radius αr, and the position of the qth virtual microphone is determined.

And Further, Ω _q = (θ _q , φ _q ) = Ω _p = (θ _p , φ _p ). In other words, the q-th (q = P + p) microphone and the p-th microphone are in the same direction as seen from the center of the virtual microphone array, the q-th microphone is on the spherical surface of radius r, and the q-th microphone The microphone is on a spherical surface with a radius αr.

Interpolation estimation section 216 includes estimated value A, virtual microphone array center D, virtual microphone P position information (r ^- _p ) (where p = 1, 2,..., P) and P pieces of positional information. Receives position information (r ^- _q ) (where q = P + 1, P + 2, ..., Q) and collects sound in the frequency domain at virtual microphone positions (r ^- _p ) and (r ^- _q ) Signal u ^ (ω, r ^- _p ) (where p = 1,2, ..., P), u ^ (ω, r ^- _q ) (where q = P + 1, P + 2, ..., Q) Is estimated (S216) and output. It should be noted that instead of P pieces of position information (r ⁻ _q ) (where q = P + 1, P + 2,..., Q), only α is received, and P pieces of position information (r ⁻ _p ) And P pieces of position information (r ⁻ _q ) may be calculated from α and α.

<Array type conversion unit 317>
The array-type conversion unit 317 is a frequency domain sound pickup signal u ^ (ω, r ⁻ _p ) (where p = 1, 2,..., P), u ^ (ω, r ⁻ _q ) (where q = P + 1, P + 2,..., Q) are received and converted into spherical wave spectra u _{n, m} (ω, r) and u _{n, m} (ω, αr) by the following equation.

In an open spherical spherical microphone array, measurement becomes impossible at k and r where j _n (kr) = 0 due to the influence of singularities. However, by selecting the larger absolute value between u _{n, m} (ω, r) and u _{n, m} (ω, αr), the double-open spherical microphone array has the effect of singularities. It can be avoided.

  Therefore, the array type conversion unit 317

Use
When | u _{n, m} (ω, r) |> | u _{n, m} (ω, αr) |

When | u _{n, m} (ω, r) | ≦ | u _{n, m} (ω, αr) |

To obtain a spherical wave spectrum v _{n, m} (ω, r).

  Finally, the array type conversion unit 317 performs inverse spherical wave spectrum conversion.

Apply. As a result, it is possible to obtain a sound collection signal in the frequency domain when a rigid spherical microphone array having a radius r is installed at the position of the double-type open spherical spherical microphone array that is virtually installed first. Array type conversion unit 317 frequency domain signal _{^{v (ω, r - p)}} ( however, p = 1,2, ..., P ) and outputs a short time inverse Fourier transform unit 218.

<Effect>
By setting it as such a structure, the effect similar to the modification 1 of 1st embodiment can be acquired. Furthermore, it is possible to virtually obtain a sound collection signal when a hard sphere type microphone array is installed. In addition, you may combine this embodiment and the modification 2 of 1st embodiment.

<Third embodiment>
Reference 4 shows an application of a hard-sphere microphone array to virtual reality.
(Reference 4) R. Duraiswami1, DN Zotkin1, Z. Li, E. Grassi, NA Gumerov, LS Davis, "High Order Spatial Audio Capture and Binaural Head-Tracked Playback over Headphones with HRTF Cues", Proceedings 119th convention of AES , 2005.
In this reference document 4, a sound signal (binaural signal) that can be heard by the right ear and the left ear when the head is turned in a specified direction with the sound pickup signal of the fixed rigid-sphere microphone array and the virtual head direction as inputs. How to output is shown. Since the spherical microphone array collects sound in all directions, a binaural signal corresponding to an arbitrary designated direction can be generated without moving the microphone element and the microphone array. That is, if the listener's head rotation is measured and input in real time, a binaural signal can be generated following the rotational movement and presented to the listener.

  In the second embodiment, a method of obtaining a sound collection signal of a virtually spherical hard sphere microphone array has been described. A configuration in which this binaural signal generation method is combined with this collected sound signal as shown in FIG. 7 is the configuration of this embodiment.

  A description will be given centering on differences from the second embodiment.

  FIG. 7 is a functional block diagram of the sound field estimation apparatus 400 according to the third embodiment, and FIG. 8 shows a processing flow thereof.

  The sound field estimation apparatus 400 includes a short-time Fourier transform unit 211, a plane wave decomposition unit 213, an interpolation estimation unit 216, an array type conversion unit 317, and a short-time inverse Fourier transform unit 218, and further includes a binaural signal generation unit 419.

<Binaural signal generator 419>
The binaural signal generation unit 419 includes a virtual head direction (posture) and a time-domain sound pickup signal y (t, r ^- _p ) (where p = 1, 2,..., P, a hard spherical spherical microphone array. And the binaural signals y (t, R), y (t, L) in the position and direction of the virtual head from these signals by the method described in Reference 4, for example. (S419) and output as the output value of the sound field estimation apparatus 400. Note that the position of the virtual head corresponds to the center D = [d _x d _y d _z ] of the virtual microphone array, and the collected sound signal y (t, r ^- _p ) in the time domain is the virtual This corresponds to the sound collection signal of the hard spherical spherical microphone array at the position of the head. Therefore, the binaural signal generation unit 419 uses the binaural signal y (() in the position and direction of the virtual head from the direction (posture) of the virtual head and the collected sound signal y (t, r ^- _p ) in the time domain. t, R), y (t, L) can be generated.

  The method of Reference 4 can only follow the rotational movement of the head and cannot cope with the translational movement of the head. However, in the configuration of the present embodiment, the rigid spherical spherical microphone array can be virtually translated. To this end, the present embodiment makes it possible to generate a binaural signal following both the rotational movement and translational movement of the head.

<Other variations>
The present invention is not limited to the above-described embodiments and modifications. For example, the various processes described above are not only executed in time series according to the description, but may also be executed in parallel or individually as required by the processing capability of the apparatus that executes the processes. In addition, it can change suitably in the range which does not deviate from the meaning of this invention.

<Program and recording medium>
In addition, various processing functions in each device described in the above embodiments and modifications may be realized by a computer. In that case, the processing contents of the functions that each device should have are described by a program. Then, by executing this program on a computer, various processing functions in each of the above devices are realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Further, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its storage unit. When executing the process, this computer reads the program stored in its own storage unit and executes the process according to the read program. As another embodiment of this program, a computer may read a program directly from a portable recording medium and execute processing according to the program. Further, each time a program is transferred from the server computer to the computer, processing according to the received program may be executed sequentially. Also, the program is not transferred from the server computer to the computer, and the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by the execution instruction and result acquisition. It is good. Note that the program includes information provided for processing by the electronic computer and equivalent to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer).

  In addition, although each device is configured by executing a predetermined program on a computer, at least a part of these processing contents may be realized by hardware.

Claims (8)

m = 1,2, ..., the index of the spherical microphone array to _{M, j m = 1,2, ...} , J m, r m a polar _radius, θ j_m and φ the polar polarization angle _{J_m,} the ω time and the frequency index, d ^- _m spheres on the J _m-number of deflection angle theta _{J_m} and phi _{J_m} position of radius r _m around the _{^{r - (m, j m)}} = d - m + [r m sinθ j_m cosφ _{j_m} r _m sinθ _{j_m sinφ j_m} r _{m cosθ j_m} ] The sound field is constructed using the collected sound signals u (ω, m, j _m ) in the frequency domain of M spherical microphone arrays m each having a microphone at ^T A plane wave decomposition unit for estimating a vector consisting of the intensity of the plane wave to be
Using the estimated value a (ω) of the vector composed of the intensity of the plane wave and the virtual microphone position r ^- _p , the sound collected signal u ^ (ω in the frequency domain at the virtual microphone position r ^- _p is used. , r ^- _p ), and an interpolation estimation unit,
Sound field estimation device. The sound field estimation apparatus according to claim 1,
α is a predetermined real number, p = 1, 2,..., P, r is the radial radius of the polar coordinate, θ _p and φ _p are the polar coordinates, and the collected sound signal u ^ (ω, r, θ _p , φ _p ) Spherical wave spectrum u _{n, m} (ω, r) absolute value | u _{n, m} (ω, r) | and spherical wave spectrum of collected sound signal u ^ (ω, αr, θ _p , φ _p ) _{u n, m (ω, αr} ) the absolute value of _{| u n, m (ω,} αr) | based on magnitude relationship between the spherical wave spectrum is calculated from the sound signals picked up by a rigid ball-type microphone array v _{n, m} ( including an array transform that estimates ω, r),
Sound field estimation device. The sound field estimation apparatus according to claim 2, wherein the position of the virtual head is defined as the position of the hard sphere microphone array.
Based on the P time collected sound signals y (t, r ^- _p ) obtained from the spherical wave spectrum v _{n, m} (ω, r) and the direction of the virtual head, Including a binaural signal generator for generating binaural signals in position and direction,
Sound field estimation device. The sound field estimation apparatus according to any one of claims 1 to 3,
P is an integer greater than or equal to 1, p = 1,2, ..., P, a (ω) = [a ₁ (ω), a ₂ (ω), ..., a _l (ω), ..., a _{L '} (ω )] ^T and i are imaginary numbers, k is a wave number, polar argument of the l-th plane wave constituting the sound field is θ _l and φ _l , and the interpolation estimation unit includes:

To estimate the sound collection signal u ^ (ω, r ^- _p ) in the frequency domain of the virtual microphone with declination θ _p and φ _p on a sphere of radius r centered at [d _x d _y d _z ] To
Sound field estimation device The sound field estimation apparatus according to any one of claims 1 to 4,
The M spherical microphone arrays m are open-type microphone arrays,
|| a (ω) || ₁ is the L1 norm of a (ω), the constant for regularization is λ, and the plane wave decomposition unit is

Is used as a cost function, and an estimated value a ⁻ (ω) that minimizes the cost function J is obtained.
Sound field estimation device. The sound field estimation apparatus according to any one of claims 1 to 4,
The M spherical microphone arrays are rigid spherical microphone arrays,
|| a (ω) || ₁ is the L1 norm of a (ω), λ is a constant for regularization, Y _n ^{m ′} is a spherical harmonic function of order n, order m ′, k is a wave number, b _n (kr _m ) is a mode intensity function, and the plane wave decomposition unit is

Is used as a cost function, and an estimated value a ⁻ (ω) that minimizes the cost function J is obtained.
Sound field estimation device. m = 1,2, ..., the index of the spherical microphone array to _{M, j m = 1,2, ...} , J m, r m a polar _radius, θ j_m and φ the polar polarization angle _{J_m,} the ω time an index of the frequency, the plane wave decomposition section, d ^- the argument of the sphere radius r _m around the _{m θ j_m} and phi _{J_m} of J _m-number of position _{^{r - (m, j m)}} = d - m + [r _m sinθ _{j_m} cosφ _{j_m} r _m sinθ _{j_m sinφ j_m} r _{m cosθ j_m} ] Using the collected sound signals u (ω, m, j _m ) in the frequency domain of M spherical microphone arrays m each having a microphone in ^T A plane wave decomposition step for estimating a vector composed of the intensity of the plane wave constituting the sound field;
Interpolation estimation unit, and estimates a (omega) of the vector of intensity of the plane wave, the position r of the virtual microphone ^- with a _p, the position r of the virtual microphone ^- sound pickup frequencies in _{the p} region An interpolation estimation step for estimating the signal u ^ (ω, r ^- _p ),
Sound field estimation method.   The program for functioning a computer as a sound field estimation apparatus in any one of Claims 1-6.

Images