JP5172909B2 - Reflected sound information estimation apparatus, reflected sound information estimation method, program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technology to estimate reflection sound information from a sound collection signal. <P>SOLUTION: Template information which is a collection of templates (TP) to show the transfer characteristic for every frequency between a pth (1&le;p&le;P) position and each position of M microphones is prepared beforehand. An observation signal and the template information are used to determine a pth complex amplitude so that the power of a residual signal obtained by subtracting from the observation signal a pth reflection sound which is a pth TP multiplied by the pth complex amplitude becomes the minimum value. The power of a residual signal obtained by subtracting from the observation signal the pth reflection sound which is the determined pth complex amplitude multiplied by the pth TP is sought regarding each p. The TP that gives the minimum power of them is determined, and near a direction D defined by the position corresponding to the determined TP, the direction D is corrected so that the power of a residual signal E obtained by subtracting from the observation signal a transfer characteristic function multiplied by the complex amplitude becomes the minimum value in order that the arrival direction of the reflection sound is estimated. <P>COPYRIGHT: (C)2012,JPO&amp;INPIT

Description

  The present invention relates to a technique for estimating information (arrival amplitude, arrival direction) relating to reflected sound from a collected sound signal obtained by collecting a sound signal with a microphone.

  A system for exchanging voice information such as telephone calls and voice conferences is generally called a voice communication system. In a voice communication system, it is very important to obtain information about the reflected sound (arrival amplitude, arrival direction, etc.). In a reverberant environment such as a conference room, the collected sound signal collected through the microphone is reflected not only from the direct sound coming from the sound source such as the speaker but also from the floor, wall or ceiling. Reflected sound is mixed. Therefore, when a speaker's utterance is recorded in such a reverberant environment, the reflected sound is mixed with a delay from the direct sound, making it difficult to hear. If arrival information of each reflected sound can be estimated from the collected sound signal and the reflected sound can be removed, it is possible to recover a sound that is easy to hear. Here, Non-Patent Document 1 is given as a conventional study for estimating reflected sound information.

  A functional configuration for realizing the technology disclosed in Non-Patent Document 1 is shown in FIG. The processing procedure in this technique is as follows.

1. The sound source signal radiated from the impulse sound source 100 is picked up using 4ch microphones 110-1, 110-2, 110-3, 110-4. The AD converter 120 converts the collected analog signal into a digital signal x ^→ (t) = [x ₁ (t), x ₂ (t), x ₃ (t), x ₄ (t)] ^T . Here, [•] ^T represents transposition. t represents a discrete time index. Assume that four microphones are arranged at the apexes of a regular tetrahedron.

2. The impulse response calculation unit 130 receives the digital signal x ^→ (t) = [x ₁ (t), x ₂ (t), x ₃ (t), x ₄ (t)] ^T as an input, and the impulse response of each microphone. h ^→ (t) = [h ₁ (t), h ₂ (t), h ₃ (t), h ₄ (t)] ^T is calculated. The impulse response calculation method includes a TSP method, an M-sequence method, and the like, and any method may be used to calculate the impulse response.

3. The virtual sound source calculation unit 140 receives 4ch impulse response h ^→ (t) = [h ₁ (t), h ₂ (t), h ₃ (t), h ₄ (t)] ^T as input, and generates virtual sound source information. v ^→ = [v ^→ ₁ ,…, v ^→ _D ] ^T is output. D represents the number of virtual sound sources. A virtual sound source is a sound source that is virtually present to represent the arrival amplitude, arrival direction, and arrival time of each reflected sound. The virtual sound source will be described with reference to FIG. FIG. 2 shows a path for receiving a sound source signal reflected by the right wall with a microphone. The sound source signal reflected from the right wall (reflected sound) is equivalent to the signal coming directly from the position written as “virtual sound source” (however, the effects of attenuation and distance attenuation due to reflection on the wall are not receive).

Details of this prior art will be described. When the impulse response is measured at four adjacent sound receiving points (microphone positions), there is a slight difference in the arrival time of the reflected sound. By using the cross-correlation of the short section of the impulse response and associating the reflected sound with each microphone, as shown in FIG. 3, the arrival time t _1n at each sound receiving point regarding the nth reflected wave, t _2n , t _3n , t _4n (1 ≦ n ≦ D) are obtained. Each virtual sound source information v _n ^→ = [X _n , Y _n , Z _n , S _n ] ^T is obtained when the length of the side of the regular tetrahedral microphone array is d and the speed of sound is c. Here, X _n , Y _n , and Z _n represent the position of the n-th virtual sound source (see equations (1) to (3)), which has information corresponding to the arrival direction and arrival time of each reflected sound. . Further, S _n represents the strength of the n-th virtual source, determined by the average of the amplitudes of the an n-th reflected sound associated with impulse 4ch.

Yoshio Yamazaki et al., “Spatial information obtained from impulse responses of four adjacent points”, Proc. Of the Acoustical Society of Japan, 1981, pp.759-760.

  According to the prior art, in order to estimate the “arrival amplitude”, “arrival direction”, and “arrival time” of the reflected sound called virtual sound source information, it is necessary to prepare an impulse response in advance. However, since it is necessary to observe using a special signal in order to prepare an impulse response, the condition that impulse responses at all positions are prepared in advance is not realistic.

  Therefore, an object of the present invention is to provide a technique for estimating reflected sound information ("arrival direction" or "arrival amplitude" of reflected sound) from a collected sound signal without using a special signal.

  A template representing transfer characteristics for each frequency between the p-th position and each position where M microphones are arranged, where P is a predetermined integer of 2 or more and p is an integer of 1 to P. Template information that is a set is prepared in advance. Using the signal (observation signal) obtained by collecting the M sound pickup signals obtained by collecting the sound signals with M microphones into the frequency domain (observation signal) and the template information, (1) p is added to the pth template. The p-th complex amplitude is determined so that the power of the residual signal obtained by subtracting the p-th reflected sound represented by multiplying the th-th complex amplitude from the observation signal is minimized, and the determined p-th complex amplitude is obtained. The power of the residual signal obtained by subtracting the p-th reflected sound expressed by multiplying the amplitude by the p-th template from the observed signal is obtained for each p, and the template that gives the minimum power among these is determined. (2) A function (transfer characteristic function) simulating the transfer characteristic for each frequency between an arbitrary position in the space and each microphone in the vicinity of the direction D determined by the position corresponding to the determined template. Estimating the arrival direction of the reflected sound by the power of the residual signal E obtained by subtracting the multiplied by the amplitude from the observed signal to correct the direction D so as to minimize.

  A template representing transfer characteristics for each frequency between the p-th position and each position where M microphones are arranged, where P is a predetermined integer of 2 or more and p is an integer of 1 to P. Template information that is a set is prepared in advance. Using the signal (observation signal) obtained by collecting the M sound pickup signals obtained by collecting the sound signals with M microphones into the frequency domain (observation signal) and the template information, (1) p is added to the pth template. The p-th complex amplitude is determined so that the power of the residual signal obtained by subtracting the p-th reflected sound represented by multiplying the th-th complex amplitude from the observation signal is minimized, and the determined p-th complex amplitude is obtained. The power of the residual signal obtained by subtracting the p-th reflected sound expressed by multiplying the amplitude by the p-th template from the observed signal is obtained for each p, and the template that gives the minimum power among these is determined. (2) A function (transfer characteristic function) simulating the transfer characteristic for each frequency between an arbitrary position in the space and each microphone in the vicinity of the direction D determined by the position corresponding to the determined template. The direction of arrival of the reflected sound is estimated by correcting the direction D so that the power of the residual signal E obtained by subtracting the amplitude multiplied from the observation signal is minimized, and the transmission corresponding to the direction of arrival is estimated. The complex amplitude multiplied by the characteristic function is estimated as the arrival amplitude of the reflected sound.

  A template representing a transfer characteristic for each frequency between the p-th position and each position where M microphones are arranged, where P is a predetermined integer of 2 or more and p is an integer satisfying 1 ≦ p ≦ P. Template information that is a set of is prepared in advance. Using a signal (observation signal) obtained by collecting the M sound pickup signals obtained by collecting the sound signals with M microphones into the frequency domain (observation signal) and template information, Q is set to 1 or more in advance. An integer, q is an integer between 1 and Q, and for each q, (1) the pth reflected sound represented by multiplying the pth template by the pth complex amplitude is the qth smallest residual signal. The p-th complex amplitude is determined so that the power of the residual signal obtained by subtracting from the first minimum residual signal is an observation signal, and the determined p-th complex amplitude is determined. The power of the q + 1th residual signal obtained by subtracting the pth reflected sound represented by multiplying the pth template from the qth minimum residual signal is obtained for each p, and the minimum power of these is obtained. Determine the template given ( ) In the vicinity of the direction D determined by the position corresponding to the determined template, a function (transfer characteristic function) simulating the transfer characteristic for each frequency between an arbitrary position in the space and each microphone was multiplied by the complex amplitude. The direction of arrival of the reflected sound is estimated by correcting the direction D so that the power of the residual signal E obtained by subtracting the signal from the qth minimum residual signal is minimized.

  A template representing a transfer characteristic for each frequency between the p-th position and each position where M microphones are arranged, where P is a predetermined integer of 2 or more and p is an integer satisfying 1 ≦ p ≦ P. Template information that is a set of is prepared in advance. Using a signal (observation signal) obtained by collecting the M sound pickup signals obtained by collecting the sound signals with M microphones into the frequency domain (observation signal) and template information, Q is set to 1 or more in advance. An integer, q is an integer between 1 and Q, and for each q, (1) the pth reflected sound represented by multiplying the pth template by the pth complex amplitude is the qth smallest residual signal. The p-th complex amplitude is determined so that the power of the residual signal obtained by subtracting from the first minimum residual signal is an observation signal, and the determined p-th complex amplitude is determined. The power of the q + 1th residual signal obtained by subtracting the pth reflected sound represented by multiplying the pth template from the qth minimum residual signal is obtained for each p, and the minimum power of these is obtained. Determine the template given ( ) In the vicinity of the direction D determined by the position corresponding to the determined template, a function (transfer characteristic function) simulating the transfer characteristic for each frequency between an arbitrary position in the space and each microphone was multiplied by the complex amplitude. The arrival direction of the reflected sound is estimated by correcting the direction D so that the power of the residual signal E obtained by subtracting the signal from the q-th minimum residual signal is minimized, and corresponds to the arrival direction. The complex amplitude multiplied by the transfer characteristic function is estimated as the arrival amplitude of the reflected sound.

  The power of the residual signal may be a power obtained by adding over all frequencies. At this time, the arrival direction is estimated by correcting the direction D so that the power of the residual signal E obtained by addition over all frequencies is minimized.

Also, ω is the frequency, Ω is the set of frequencies ω, i is the imaginary unit, c is the speed of sound, the p th position [x _p , y _p , z _p ] and the m th (1 ≦ m ≦ M) microphone are arranged. S _pm (ω), the transfer characteristic between the position [u _m , v _m , w _m ]
, Template information {S ₁ (ω), ..., S _P (ω)} that is a set of templates S _p (ω) = {S _p1 (ω), ..., S _pM (ω)} (ω∈Ω) A template generation process for generating (ω∈Ω) may be included.

For example, the transfer characteristic function is a transfer function for each frequency between an arbitrary position [x, y, z] in space and each position [u _m , v _m , w _m ] where M microphones are arranged. The transfer characteristic R _m (ω) is expressed by the characteristic R _m (ω) (1 ≦ m ≦ M), where the frequency is ω, i is an imaginary unit, and c is the speed of sound.
It is represented by

  According to the present invention, template information that is a set of templates representing transfer characteristics for each frequency between a space (or plane) position and each position where M microphones are arranged is created in advance. It is possible to estimate the reflected sound information from the collected sound signal without using a special signal for the sound source signal to obtain the impulse response because the observation signal is decomposed into one or more reflected sounds based on is there. When the reflected sound information is obtained, it can be applied to applications such as estimation of the sound source direction and voice enhancement (collection of far-field sounds and sound collection by distance) that could not be realized by conventional voice information processing technology.

The figure which shows the function structure of the reflected sound information estimation technique in a prior art. The figure for demonstrating a virtual sound source. The figure for demonstrating matching of the reflected sound in a prior art. The figure which shows the function structure of the reflected sound information estimation apparatus which concerns on 1st Embodiment. The figure which shows the process sequence of the reflected sound information estimation method which concerns on 1st Embodiment. The figure which shows the structural example of a two-dimensional microphone array. p-th point _{[x p, y p, z} p] diagram for explaining the transfer characteristic between the m-th receiving point _{[u m, v m, w} m]. The figure for demonstrating that the sound pressure distribution in a certain plane observed using the microphone array shown in FIG. 6 is obtained by superimposing the direct sound, the reflected sound 1, and the reflected sound 2, for example. The figure for demonstrating the principle of this invention. The figure which shows the process sequence of the reflected sound information estimation method which concerns on 2nd Embodiment. (A) The figure for demonstrating that only the information regarding an estimated arrival direction should be extracted ideally. (B) The figure for demonstrating that the information regarding directions other than an estimated arrival direction will actually be mixed. The figure for demonstrating reducing the influence of directions other than an estimated arrival direction by summarizing the power of a residual signal over all the frequencies. The figure which shows the sound pressure distribution at the time of using a two-dimensional matrix microphone array of a practical use level, and its decomposition | disassembly.

<< First Embodiment >>
In the present invention, a voice signal (sound source signal) radiated from a sound source such as an utterance signal is collected by a microphone array composed of a plurality of microphones (sound collection signal), or “arrival direction” of reflected sound, Estimate the “arrival amplitude” and “arrival direction”. The functional configuration and processing flow of the first embodiment are shown in FIGS.

The sound source signal radiated from the sound source 200 is collected using the Mch microphones 210-1,..., 210-M (step S1). A value greater than 4 is desirable for M. The AD converter 220 converts the collected analog signal into a digital signal xx ^→ (t) = [xx ₁ (t),..., Xx _M (t)] ^T (step S2). Here, [•] ^T represents transposition. t represents a discrete time index.

It is desirable to arrange the M microphones at equal intervals in two dimensions or three dimensions. This is to uniquely determine the correspondence between the arrival direction of the reflected sound and the template (which will be described later, which simulates the transfer characteristic of the reflected sound). In principle, the present invention can be implemented even if the microphones are arranged one-dimensionally or not at regular intervals, but there is a one-to-one relationship between the transmission characteristics of the reflected sound and the arrival directions of the reflected sound. Therefore, it is desirable to arrange them at equal intervals in two dimensions or three dimensions. An example when microphones are arranged at equal intervals on a two-dimensional plane is shown in FIG. The microphone interval d is preferably set so as to satisfy the spatial sampling theorem. When the spatial sampling theorem is satisfied, the microphone interval d is a numerical value that satisfies Equation (4). c is the speed of sound, and f is the frequency to be analyzed. For example, when analyzing a frequency of 4 kHz, it is preferable to set the microphone interval to about 4 cm.

The frame division unit 230 receives the digital signal xx ^→ (t) = [xx ₁ (t),..., Xx _M (t)] ^T output from the AD conversion unit 200, and is a digital signal composed of a plurality of samples for each channel. The signal x ^→ (k) = [x ₁ (k),..., X _M (k)] ^T divided into groups (frames) is output (step S3). k is an index representing a frame number. The frame division is a process of buffering and outputting W points for each digital signal xx _i (t) (1 ≦ i ≦ M) of each channel. W depends on the sampling frequency, but in the case of 16 kHz sampling, around 512 points are appropriate.

The frequency domain converter 240 receives the digital signal x ^→ (k) of each frame as an input, and the frequency domain signal X ^→ (ω, k) = [X ₁ (ω, k),..., X _M (ω, k )] Converted to ^T and output (step S4). This signal X ^→ (ω, k) is called an observation signal. Here, ω indicates a discrete frequency index (there is a relationship of ω = 2πf between the frequency f and the angular frequency ω, and therefore the frequency index ω may be identified with the angular frequency ω. Hereinafter, with respect to ω, “frequency index” is also simply referred to as “frequency”), and k indicates a frame index. One method for transforming into the frequency domain is discrete Fourier transform, but other methods may be used as long as the transform is performed into the frequency domain. The observation signal X ^→ (ω, k) in the frequency domain is output for each frequency ω and frame k.

The template generation unit 250 includes template information S ^→ (ω) = [S ₁ ^→ (ω),..., Which is a set of P templates _Sp ^→ (ω) (however, for convenience of calculation, expressed in vector). , S _P ^→ (ω)] ( ^∀ ω∈Ω; Ω is a set of frequency indices ω) is generated for each frequency ω (step Sp). This process is usually performed prior to each process of steps S1-S4. P represents the total number of templates, and is set in advance to an integer value of 2 or more. The larger the total number of templates P, the more accurately the reflected sound information is estimated. However, since the amount of calculation increases, for example, P = 1000 is preferable. This processing is performed in advance before observing the signal with the microphone. Further, unless the position of the microphone (for example, the distance d between the microphones) is changed or the total number P of templates is not changed, it is usually unnecessary to recreate the template every time. The “template” referred to here is a simulation of a transfer characteristic (acoustic propagation characteristic) corresponding to the arrival direction of the reflected sound. The p-th (1 ≦ p ≦ P) template S _p ^→ (ω) = [S _p1 (ω),..., S _pM (ω)] ^T (ω∈Ω) is a predetermined p-th point [ x _p , y _p , z _p ] and M sound receiving points (where the sound receiving point is the position where the microphone is placed, and the m th (1 ≦ m ≦ M) sound receiving point is represented by [u _m , v _m , w _m ]) (see FIG. 7). An example of a calculation formula for each element S _pm (ω) of the p-th template _Sp ^→ (ω) is shown in Formula (5). The symbol i represents an imaginary unit.

Direction information θ _p ^→ (ω) is associated with the p-th template S _p ^→ (ω). The direction information θ _p ^→ (ω) is a three-dimensional orthogonal coordinate system that serves as a reference for the position coordinates of the p th point [x _p , y _p , z _p ] and the sound receiving point [u _m , v _m , w _m ]. The direction of viewing the p-th point [x _p , y _p , z _p ] from the origin of, for example, two declinations in a spherical coordinate system (having a common origin with the origin of the 3D Cartesian coordinate system) ( _Polar angle θ _{p, pol} and azimuth angle θ _{p, azi} ). That is, θ _p ^→ (ω) = [θ _{p, pol} (ω), θ _{p, azi} (ω)]. If the p-th template S _p ^→ (ω) is associated with the p-th point [x _p , y _p , z _p ], the direction information θ _p ^→ (ω) is the position [x _p , y _p , since can be calculated from z _p], p-th template S _p ^→ (ω) be the direction information θ _p ^→ (ω) is associated with is not a requirement. Since the three-dimensional orthogonal coordinate system and the spherical coordinate system can be converted to each other (coordinate conversion), the right side of Equation (5) is not the position [x, y, z] but the direction information θ _p ^→ (ω). = [Θ _{p, pol} (ω), θ _{p, azi} (ω)], for example, can also be expressed as shown in equation (5a). Here, d is a microphone interval, the microphone array is a two-dimensional microphone array of Φ rows and Ξ columns (Φ × Ξ = M), and the position of the mth microphone is φ rows and ξ columns (1 ≦ φ ≦ Φ, 1 ≦ ξ ≦ Ξ).

Further, when the template corresponds to the direction as in the first embodiment, the positions of the P points [x _p , y _p , z _p ] (1 ≦ p ≦ P) are positions having different directions from each other. For example, assuming that each point [x _p , y _p , z _p ] is at an equal distance sufficiently away from the origin, different P points on the sphere centered on the origin may be used. The reason why each point [x _p , y _p , z _p ] is located sufficiently away from the origin is that the signal radiated from the sound source or virtual sound source is transmitted spherically but sufficiently separated from the sound source or virtual sound source This is because a direct sound or reflected sound can be simulated as a plane wave in a local region at the position (origin). However, this does not mean that the template information includes a template corresponding to a position in the same direction. It is assumed that the microphone array is disposed in the vicinity (local region) of the origin of the coordinate system.

The template storage unit 260 stores the template information S ^→ (ω) output from the template generation unit 250 and plays a role of providing the template information S ^→ (ω) to the reflected sound information estimation unit 270 at the time of analysis.

The reflected sound information estimation unit 270 receives a frequency domain observation signal X ^→ (ω, k) and template information S ^→ (ω) as an input, and a set of Q reflected sound information components rs _q ^→ (ω, k) ( However, the reflected sound information rs ^→ (ω, k) = [rs ₁ ^→ (ω, k),..., Rs _Q ^→ (ω, k)] ^T is calculated for each frame. k is output for each frequency ω (step S5). Here, Q represents the total number of reflected sounds to be estimated, and is set in advance to an integer value of 1 or more. The q-th (1 ≦ q ≦ Q) reflected sound information component rs _q ^→ (ω, k) is expressed as rs _q ^→ (ω, k) = [rsA _q (ω, k), rsB _q (ω, k)] RsA _q (ω, k) is the arrival amplitude of the qth reflected sound, and rsB _q (ω, k) is the arrival direction of the qth reflected sound.

  The principle of estimating the reflected sound information will be described. An example of the sound pressure distribution in a certain plane observed using a two-dimensional microphone array as shown in FIG. 6 is shown as a shading diagram at the left end of FIG. Regarding the view of the sound pressure distribution shown as a shading diagram, the black portion indicates that the sound pressure is low and the white portion indicates that the sound pressure is high. The observed sound pressure distribution includes not only the sound pressure distribution of the direct sound but also the sound pressure distribution of the reflected sound. When the direct sound and the reflected sound come sufficiently far away, each sound pressure distribution on the two-dimensional plane has a striped pattern as shown in the three shades on the right side of FIG. Striped “shading” corresponds to the arrival amplitude of direct sound or reflected sound, and “rotation / period” corresponds to the direction of arrival of direct sound or reflected sound. In the example of FIG. 8, it is shown that the sound pressure distribution of the observation signal is configured by superimposing the sound pressure distributions of the direct sound, the reflected sound 1 and the reflected sound 2 having different arrival amplitudes and directions. When considered in the frequency domain, the direct sound and each reflected sound are represented by a complex sine wave whose frequency changes according to the direction of arrival, and the observation signal is a superposition of the direct sound and multiple complex sine waves corresponding to each reflected sound. Represented as: Incidentally, the problem to be solved by the present invention is to estimate the arrival amplitude and / or the arrival direction of the reflected sound using only the observation signal. The solution to this problem is to estimate the “tone” and “rotation / cycle” of the striped pattern corresponding to the direct sound and the reflected sounds of the three shades on the right side of FIG. 8 from the sound pressure distribution drawn at the left end of FIG. Corresponding to.

With reference to FIG. 9, an outline of a method for estimating the reflected sound information rs ^→ (ω, k) will be described. The reflected sound 0 with the strongest power included in the observation signal observed on a two-dimensional plane (corresponding to q = 1 and having the strongest power, this reflected sound 0 is usually understood as a “direct sound”. to) estimates to obtain a residual signal E ₂ from the observed signal by subtracting the reflected sound 0. Next, the reflected sound 1 having the strongest power (corresponding to q = 2) included in the residual signal E ₂ is estimated, and the reflected sound 1 is subtracted from the residual signal E ₂ to obtain a new residual sound. obtaining a difference signal E _3. Next, the reflected sound 2 having the strongest power (corresponding to q = Q = 3) included in the residual signal E ₃ is estimated. Here, the case of Q = 3 has been described, but generally, the qth strongest power included in the _qth residual signal E _q (where the first residual signal is an observed signal) The reflected sound information components (rs ₁ ^→ (ω, k),..., Are sequentially executed by subtracting the reflected sound q-1 (where reflected sound 0 is a direct sound) until q = Q. rs _Q ^→ (ω, k)). The first reflected sound information component rs ₁ ^→ (ω, k) corresponds to the reflected sound 0 (direct sound), the second reflected sound information component rs ₂ ^→ (ω, k) corresponds to the reflected sound 1, The third reflected sound information component rs ₃ ^→ (ω, k) corresponds to the reflected sound 2..., The Qth reflected sound information component rs _Q ^→ (ω, k) corresponds to the reflected sound Q-1. To do. Q depends on the application using the calculation power and the reflected sound information, but is preferably set to about 30.

Note that the sound pressure distributions in FIGS. 8 and 9 are shown as high-resolution shading diagrams, but in order to show the sound pressure distribution as such a high-resolution shading diagram, an extremely large number of microphones are required, Not practical. On the other hand, even when, for example, 100 microphones are used as a 10 × 10 two-dimensional matrix microphone array as a practical level two-dimensional matrix microphone array, the sound pressure shown as a rough (low resolution) gray scale (see FIG. 13). Only a distribution can be obtained. Thus, from a practical point of view, it is required to accurately estimate the arrival amplitude and direction of the reflected sound in a situation where only a low-resolution sound pressure distribution can be obtained. In the present invention, a plane wave arriving from an arbitrary position is specifically expressed in order to improve spatial resolution (formulation), and reflected sound with low power can be estimated under the influence of reflected sound with high power. In order to prevent disappearance, the already estimated reflected sound is removed from the observation signal to estimate the next reflected sound (decomposition). The formulation is as described in the template information, and the decomposition is as described in the outline of the estimation method of the reflected sound information rs ^→ (ω, k).

The method for estimating the reflected sound information rs ^→ (ω, k) described above will be described in detail. Prior to explanation, symbols are defined. q q residual signal is represented by E _q ^→ (ω, k) = [E _q1 (ω, k), ..., E _qM (ω, k)] ^T , qth reflected sound (if q = 1, direct A _q (ω, k) R _q ^→ (ω, θ _q ^→ (ω, k)). R _q ^→ (ω, θ _q ^→ (ω, k)) = (R ₁ (ω, θ _q ^→ (ω, k)), ..., R _M (ω, θ _q ^→ (ω, k))] ^T is a function that simulates the transfer characteristics for each frequency between an arbitrary position [x, y, z] in space and each microphone (hereinafter referred to as the transfer characteristic function). Any function that simulates the transfer characteristics may be used. The reason why such a transfer characteristic function is a component of the reflected sound is that a template corresponding to the direction considered to be closest to the direction of arrival of the reflected sound is determined and the direction D corresponding to the template is determined. This is to improve the estimation accuracy of the arrival direction of the reflected sound by correcting the direction D in the vicinity (the details are reflected sound A _q (ω, k) R _q ^→ (ω, θ _q ^→ (ω, It will be described later as optimization of k))). Usually, each transfer characteristic R _m (ω, θ _q ^→ (ω, k)) constituting the transfer characteristic function and the calculation formula for each element S _pm (ω) of the template are the same. In this case, between the position [x, y, z] in the direction represented by the direction information θ _q ^→ (ω, k) and the m th sound receiving point [u _m , v _m , w _m ] The transfer characteristic R _m (ω, θ _q ^→ (ω, k)) for each frequency is expressed by equation (6). Note that the position [x, y, z] in the direction represented by the direction information θ _q ^→ (ω, k) may be, for example, a position on a spherical surface sufficiently away from the coordinate system origin. The reason why the position [x, y, z] is sufficiently distant from the origin is as described above. Specifically, the position [x, y, z] is a sound source in the local region where the microphone array is arranged. Or it is preferably an arbitrary position in a space at a distance that can simulate a direct sound or reflected sound from a virtual sound source as a plane wave. Since the three-dimensional orthogonal coordinate system and the spherical coordinate system can be converted to each other (coordinate conversion), the right side of Equation (6) is not the position [x, y, z] but the direction information θ _q ^→ (ω, Using k) = [θ _{q, pol} (ω, k), θ _{q, azi} (ω, k)], for example, it can also be expressed as in equation (6a). Here, d is a microphone interval, the microphone array is a two-dimensional microphone array of Φ rows and Ξ columns (Φ × Ξ = M), and the position of the mth microphone is φ rows and ξ columns (1 ≦ φ ≦ Φ, 1 ≦ ξ ≦ Ξ).

A _q (ω, k) constituting the reflected sound is a template R _q ^→ (ω, θ _q ^→ (ω, k)) such as the phase of the sound source 200 itself, reflection on the wall, attenuation due to distance, and reflected sound. This corresponds to the arrival amplitude. The above-described method for subtracting the reflected sound from the residual signal in ascending order of q is expressed by equation (7). However, 1 ≦ q ≦ Q, and E ₁ ^→ (ω, k) = X ^→ (ω, k).

Next, a method for optimizing the reflected sound A _q (ω, k) R _q ^→ (ω, θ _q ^→ (ω, k)) will be described.
The q-th optimized reflected sound A _q (ω, k) R _q ^→ (ω, θ _q ^→ (ω, k)) is expressed as q + 1-th residual signal E _{q +} expressed by the equation (7). ₁ ^→ (ω, k) power (E _{q + 1} ^→ (ω, k)) ^H E _{q + 1} ^→ (ω, k) is determined according to a standard that minimizes the power. Specifically, when it is noted that the transfer characteristic function R _q ^→ (ω, θ _q ^→ (ω, k)) is determined by the direction information θ _q ^→ (ω, k), the q-th reflected sound A _q (ω, k) R _q ^→ (ω, θ _q ^→ (ω, k)) parameter A _q (ω, k), θ _q ^→ (ω, k) optimum value A _{q, opt} (ω, k), θ _{q, opt} ^→ (ω, k) is obtained by equation (8). Note that the symbol H represents conjugate transposition.

At this time, the q-th reflected sound information component rs _q ^→ (ω, k) = [rsA _q (ω, k), rsB _q (ω, k)] is given by equations (9) and (10).

  Various specific calculation methods of the formula (8) are conceivable, but an example is shown here. The optimization method described below is applied to each q in ascending order of q.

§1 Initial value setting of direction information First, initial value θ _{ini, q} ^→ (ω, k) of direction information θ _q ^→ (ω, k) is determined using template information S ^→ (ω). For this purpose, a template corresponding to the direction considered to be closest to the direction of arrival to be estimated is determined, and the direction information corresponding to the determined template is determined as the initial value θ of the direction information θ _q ^→ (ω, k). _{ini, q} ^→ (ω, k).

Therefore, in order to determine the template as described above from the template information, the reflected sound is expressed as A _q (ω, k, g (ω, q)) S _{g (ω, q)} ^→ (ω) for convenience. I will decide. Here, g (ω, q) represents an index of a template that can represent the q-th reflected sound most accurately in the template information. The coefficient A _q (ω, k, g (ω, q)) constituting the reflected sound is a template S _{g (ω, q)} ^→ (ω due to the phase of the sound source 200 itself, reflection on the wall, attenuation due to distance, etc. ) And the reflected sound. In this case, the q + 1-th residual signal E _{q + 1} ^→ (ω, k) is expressed as in Expression (11). However, E ₁ ^→ (ω, k) = X ^→ (ω, k).

The reflected sound A _q (ω, k, g (ω, q)) S _{g (ω, q)} ^→ (ω) is expressed as q + 1th residual signal E _{q + 1} ^→ (ω, k) based on the equation (11). ) Power (E _{q + 1} ^→ (ω, k)) ^H E _{q + 1} ^→ (ω, k) is estimated according to a standard that minimizes. There are various estimation methods, but one of them will be described. The reflected sound consists of two elements, A _q (ω, k, g (ω, q)) and S _{g (ω, q)} ^→ (ω). Is required. <Process 1> and <Process 2> to be described later are performed for each q in ascending order of q.

<Process 1>
The symbol Λ is a set obtained by excluding the set of indexes determined by Equation (13) described later from the entire set {1,..., P,. That is, Λ = {1, ..., p, ..., P}-{g (ω, 1), ..., g (ω, q-1)}. However, when <Process 1> is performed for the first time, Λ = {1,..., P,.
The p-th template S _p ^→ (ω) is the power (E _{q + 1} ^→ (ω, k)) ^H E _{q + 1} ^→ (ω, k) of the residual signal E _{q + 1} ^→ (ω, k) The coefficient A _q (ω, k, p) when it is assumed that the template is the optimum template for minimization is obtained by Expression (12) based on the least square method. Note that at this stage, q on the left side of Equation (9) has no meaning.

<Process 2>
If the number (concentration) of elements of the set Λ is | Λ |, the | Λ | coefficients A _q (ω, k, p) (p∈Λ) obtained based on the equation (12) are used to generate a template. _{G (ω, q)} representing the index of S _{g (ω, q)} ^→ (ω) is the power of the residual signal E _{q + 1} ^→ (ω, k) (E _{q + 1} ^→ (ω, k)) ^As an index that minimizes ^H E _{q + 1} ^→ (ω, k), it is obtained by Expression (13).

Therefore, the initial value θ _{ini, q} ^→ (ω, k) of the direction information θ _q ^→ (ω, k) is a template S _{g (ω,} k) having g (ω, q) obtained by the equation (13) as an index _{. q)} ^→ Direction information corresponding to (ω) θ _{g (ω, q)} ^→ (ω) = [θ _{g (ω, q), pol} (ω), θ _{g (ω, q), azi} (ω)] As given. That is, θ _{ini, q} ^→ (ω, k) = [θ _{g (ω, q), pol} (ω), θ _{g (ω, q), azi} (ω)]. Note that the initial value θ _{ini, q} ^→ (ω, k) does not depend on the frame index k.

§2 Optimization of reflected sound Next, starting from the initial value θ _{ini, q} ^→ (ω, k) of the direction information θ _q ^→ (ω, k), the q + 1th residual represented by equation (7) In order to minimize the power (E _{q + 1} ^→ (ω, k)) ^H E _{q + 1} ^→ (ω, k) of the signal E _{q + 1} ^→ (ω, k), the reflected sound A _q (ω, k) k) Optimize R _q ^→ (ω, θ _q ^→ (ω, k)). The reflected sound is composed of two elements: coefficient A _q (ω, k) and R _q ^→ (ω, θ _q ^→ (ω, k)), so it is necessary to optimize for the two elements It becomes. There are various optimization methods, but one of them (gradient method) will be described. In the illustrated method, correction of the direction information θ _q ^→ (ω, k) and correction of the coefficient A _q (ω, k) are alternately repeated a predetermined number of times (δ times), whereby the reflected sound A _q (ω , k) R _q ^→ (ω, θ _q ^→ (ω, k)) is optimized. For example, δ is about 50, but may be 1.

§2.1 Correction of direction information Correction of direction information θ _q ^→ (ω, k) = [θ _{q, pol} (ω, k), θ _{q, azi} (ω, k)] is updated by equation (14) Is done by. When the processing of §2.1 is performed for the first time, the direction information θ _q ^→ (ω, k) on the right side of Expression (14) is the initial value θ _{ini, q} ^→ (ω, k) obtained by the processing of §1. If the processing in §2.1 is not the first time, the direction information θ _q ^→ (ω, k) on the right side of equation (14) is the direction information obtained in the immediately preceding processing in §2.1. Further, when the processing of §2.1 is performed for the first time, the coefficient A _q (ω, k) used for calculating the power (E _{q + 1} ^→ (ω, k)) ^H E _{q + 1} ^→ (ω, k) Is A _q (ω, k, p) obtained in Equation (12), and if the processing in §2.1 is not the first time, the power (E _{q + 1} ^→ (ω, k)) ^H E _{q + 1} ^{→ The} coefficient A _q (ω, k) used in the calculation of (ω, k) is the coefficient A _q (ω, k) obtained in the immediately preceding §2.2 process (described later). The step widths α ₁ and α ₂ are small positive constants, which are determined in consideration of the convergence speed and the like, and are each about 0.1, for example.

§2.2 Correction of coefficient The coefficient A _q (ω, k) is corrected by obtaining a new coefficient A _q (ω, k) according to the equation (15) based on the least square method. R _q ^→ (ω, θ _q ^→ (ω, k)) used in Expression (15) is obtained from the direction information θ _q ^→ (ω, k) obtained by the processing of §2.1 and Expression (6). .

The coefficient A _q (ω, k) and direction information θ _q ^→ (ω, k) obtained when δ iterations are completed are A _{q, opt} (ω, k) and θ _{q, opt} ^→ ( ω, k), and the q-th reflected sound information component rs _q ^→ (ω, k). That is, the q-th reflected sound information component rs _q ^→ (ω, k) = [rsA _q (ω, k), rsB _q (ω, k)] is given by the equations (16) and (17).

Through the above process, Q reflected sound information components rs _q ^→ (ω, k) = [rsA _q (ω, k), rsB _q (ω, k)] (q = 1,..., Q) are obtained. . When δ = 1 is set, only the direction of arrival can be obtained as reflected sound information by not correcting the coefficient.

<< Second Embodiment >>
In the first embodiment, the reflected sound information rs ^→ (ω, k) is obtained using the template information S ^→ (ω). However, the template information S ^→ (ω) is a set of P templates _Sp ^→ (ω). ) Is not necessarily required in advance. An embodiment in which template information S ^→ (ω) is not obtained in advance will be described as a second embodiment.

  In the second embodiment, the processes in steps S1 to S4 in the first embodiment are performed. However, the process in step Sp in the first embodiment is not necessary, and is further replaced with the process in step S5 in the first embodiment. Then, the process of step S5a is performed (see FIG. 10). Therefore, the duplicated description of the same items as those in the first embodiment will be omitted, and items different from those in the first embodiment will be described.

The process of step S5a in 2nd Embodiment is demonstrated. In the process of step S5a in the second embodiment, “§1 Initial value setting of direction information” is different from the first embodiment. The initial value θ _{ini, q} ^→ (ω, k) of the direction information θ _q ^→ (ω, k) is determined by an arrival direction estimation method such as a beam former method. The beam former method is a method in which a directional beam is spatially scanned and a direction in which power is increased is searched for from the obtained power spectrum. Here, it is assumed that P arrival directions can be estimated by the beamformer method.

  In practice, the power spectrum obtained by the beamformer method may not be steep with respect to the direction of arrival. In such a case, for example, the range of the direction corresponding to the power spectrum showing the spectral intensity equal to or higher than the predetermined spectral intensity. The arrival direction may be determined at predetermined intervals. As a specific example, assuming that a power spectrum showing a spectral intensity equal to or higher than a predetermined spectral intensity in a polar angle range of 5 ° and an azimuth angle of 10 ° to 20 ° is obtained, the direction of arrival is determined every predetermined interval of 2 °. (Polar angle 5 °, azimuth angle 10 °), (polar angle 5 °, azimuth angle 12 °), (polar angle 5 °, azimuth angle 14 °), (polar angle 5 °, azimuth angle 16 °), (Polar angle 5 °, azimuth angle 18 °), (polar angle 5 °, azimuth angle 20 °) may be the arrival direction.

  Even if the power spectrum shows a steep peak in a certain direction, the direction of arrival may be determined within a predetermined range of the direction instead of simply determining that direction as one of the directions of arrival. As a specific example, assuming that a power spectrum showing a steep peak at a polar angle of 30 ° and an azimuth angle of 50 ° is obtained, the direction of arrival is within a predetermined range (polar angle ± 4 °, azimuth angle ± 4 °, interval 2 °). (Polar angle 26 °, azimuth angle 46 °), (polar angle 28 °, azimuth angle 46 °), (polar angle 30 °, azimuth angle 46 °), (polar angle 32 °, azimuth angle 46 ° ), (Polar angle 34 °, azimuth angle 46 °), (polar angle 26 °, azimuth angle 48 °), (polar angle 28 °, azimuth angle 48 °), (polar angle 30 °, azimuth angle 48 °), (Polar angle 32 °, azimuth angle 48 °), (polar angle 34 °, azimuth angle 48 °), (polar angle 26 °, azimuth angle 50 °), (polar angle 28 °, azimuth angle 50 °), (polar Angle 30 °, azimuth angle 50 °), polar angle 32 °, azimuth angle 50 °, polar angle 34 °, azimuth angle 50 °, polar angle 26 °, azimuth angle 52 °, and polar angle 28 °, azimuth angle 52 °), (polar angle 30 °, azimuth 52 °), (polar angle 32 °, azimuth angle 52 °), (polar angle 34 °, azimuth angle 52 °), (polar angle 26 °, azimuth angle 54 °), (polar angle 28 °, azimuth angle 54 ° ), (Polar angle 30 °, azimuth angle 54 °), (polar angle 32 °, azimuth angle 54 °), and (polar angle 34 °, azimuth angle 54 °). Note that P is a fixed value in the first embodiment, but in the second embodiment, P is a value that depends on an estimation result obtained by an arrival direction estimation method such as a beamformer method.

A template is generated for P arrival directions obtained by the beamformer method. A calculation formula for each element of the template is, for example, Formula (6). The “§1 initial value setting of direction information” described in the first embodiment may be performed using the P templates (template information S ^→ (ω)). The processing after the initial value setting is as described in the first embodiment.

<Modification>
In the first embodiment described above, the reflected sound information rs ^→ (ω, k) is estimated using the observation signal X ^→ (ω, k) for each frequency. However, when the reflected sound information is estimated for each frequency, the reflected sound information is estimated uniquely. Information on directions other than the direction of the virtual sound source to be performed (estimated arrival direction) may be included, and as a result, errors may occur in the reflected sound information. For example, it is desirable that only information related to the estimated arrival direction can be extracted as shown in FIG. 11A, but in reality, information related to directions other than the estimated arrival direction is mixed as shown in FIG. 11B. There can be.

  Therefore, in the modification, the estimation error of the reflected sound information is reduced by calculating the power collectively over all frequencies. That is, as shown in FIG. 12, the influence of directions other than the estimated arrival direction can be reduced as much as possible by integrating the power of the residual signal over all frequencies. In general, the power at each frequency varies in directions other than the estimated direction of arrival. Therefore, by integrating the power of the residual signal over all frequencies, the power in the other direction compared to the power in the estimated direction of arrival. The relative influence of power can be reduced. In FIG. 12, it should be noted that the scale of each graph is not the same because the power on the vertical axis indicates a relative value.

The processing in this modification is as follows. A set of frequency indexes ω included in the frequency band to be analyzed is Ω. For example, if an audio signal is handled, the set of indexes corresponding to the 1.0 to 3.0 kHz band may be Ω. Then, the index g (ω, q) of the template S _{g (ω, q)} ^→ (ω) is obtained by Expression (18) instead of Expression (13). Further, the correction of the direction information θ _q ^→ (ω, k) = [θ _{q, pol} (ω, k), θ _{q, azi} (ω, k)] is performed by the equation (19) instead of the equation (14). Done by renewal.

<Application example>
The reflected sound information is very important voice information for human life. For example, a visually impaired person grasps the environment by reflecting a sound source signal generated by tapping on a wall or ceiling and observing with an ear. Further, even in everyday conversation, there is a difference in the ease of conversation between talking in a room where moderate reflection occurs and talking in an environment with relatively few reflected sounds. Hereinafter, service examples using reflected sound information estimated according to the present invention will be described.
The first is an example in which the present invention is incorporated in a conference system. Since the amplitude of the reflected sound changes according to the direction of the directional sound source, if the reflected sound information is known, it is possible to estimate in which direction the sound source is directed. If a sound source direction estimation device is incorporated in the conference system, it can be applied to presenting who spoke.
The second is a system that allows users to view video and audio at any position. Sound far away is difficult to pick up because the power of the sound source coming directly is small. If the reflected sound information is known, not only the direct sound but also the reflected sound can be picked up and collected, so that it is possible to enhance the sound in the distance. In the field of audio processing, it is possible to emphasize and collect sound sources by direction, but it is very difficult to emphasize and collect sounds by distance. If the reflected sound information is known, a physical feature amount corresponding to the distance can be obtained, so that sound can be collected for each distance. If far-field sounds can be picked up or picked up by direction and distance, a sound field corresponding to the position selected by the viewer can be generated in a pseudo manner.

  In a voice communication system, estimating reflected sound information leads to obtaining information on a sound field that could not be obtained only by direct sound. If the reflected sound information is known, it will lead to far-field sound collection and sound collection by distance, which could not be done with conventional speech enhancement technology, or information on the sound field that could not be estimated with conventional sound collection technology (for example, The direction of the sound source can be estimated. Such estimation of sound field information leads to the development of a speech processing apparatus that could not be realized by the conventional technology. The prior art related to the estimation of reflected sound information required observation of a special signal in order to obtain an impulse response, but the present invention has an advantage that reflected sound information can be obtained with a general observation signal such as an audio signal. have.

<Example of hardware configuration of reflected sound information estimation device>
The reflected sound information estimation apparatus according to the above-described embodiments may include an input unit to which a keyboard or the like can be connected, an output unit to which a liquid crystal display or the like can be connected, a CPU (Central Processing Unit) [cache memory, or the like. ] RAM (Random Access Memory) or ROM (Read Only Memory) and external storage device as a hard disk, and data exchange between these input unit, output unit, CPU, RAM, ROM, and external storage device It has a bus that can be connected. Further, if necessary, the reflected sound information estimation device may be provided with a device (drive) that can read and write a storage medium such as a CD-ROM. A physical entity having such hardware resources includes a general-purpose computer.

  The external storage device of the reflected sound information estimation device stores a program for estimating reflected sound information and data necessary for processing of the program [not limited to the external storage device, for example, a program is read-only. You may memorize | store in ROM which is a memory | storage device. ]. Data obtained by the processing of these programs is appropriately stored in a RAM or an external storage device. Hereinafter, a storage device that stores data, addresses of storage areas, and the like is simply referred to as a “storage unit”.

  The storage unit of the reflected sound information estimation device has a program for performing AD conversion on an analog signal, a program for performing frame division processing, and a program for converting a digital signal for each frame into an observation signal in the frequency domain A program for generating template information and a program for estimating reflected sound information using frequency domain observation signals and template information are stored.

  In the reflected sound information estimation apparatus, each program stored in the storage unit and data necessary for processing each program are read into the RAM as necessary, and are interpreted and processed by the CPU. As a result, the CPU realizes the predetermined functions (AD conversion unit, frame division unit, frequency domain conversion unit, template generation unit, reflection sound information estimation unit), thereby realizing the reflection sound information estimation.

<Supplementary note>
The present invention is not limited to the above-described embodiment, and can be appropriately changed without departing from the spirit of the present invention. In addition, the processing described in the above embodiment may be executed not only in time series according to the order of description but also in parallel or individually as required by the processing capability of the apparatus that executes the processing. .

  When the processing functions in the hardware entity (reflected sound information estimation apparatus) described in the above embodiment are realized by a computer, the processing contents of the functions that the hardware entity should have are described by a program. Then, by executing this program on a computer, the processing functions in the hardware entity are realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like, and as an optical disk, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only). Memory), CD-R (Recordable) / RW (ReWritable), etc., magneto-optical recording medium, MO (Magneto-Optical disc), etc., semiconductor memory, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. Can be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. When executing the process, the computer reads a program stored in its own recording medium and executes a process according to the read program. As another execution form of the program, the computer may directly read the program from a portable recording medium and execute processing according to the program, and the program is transferred from the server computer to the computer. Each time, the processing according to the received program may be executed sequentially. Also, the program is not transferred from the server computer to the computer, and the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by the execution instruction and result acquisition. It is good. Note that the program in this embodiment includes information that is used for processing by an electronic computer and that conforms to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer).

  In this embodiment, a hardware entity is configured by executing a predetermined program on a computer. However, at least a part of these processing contents may be realized by hardware.

Claims (15)

A template representing transfer characteristics for each frequency between the p-th position and each position where M microphones are arranged, where P is a predetermined integer of 2 or more and p is an integer of 1 to P. A storage unit for storing template information that is a set;
A signal obtained by collecting audio signals with M microphones and obtained by converting the M sound collection signals into the frequency domain (hereinafter referred to as an observation signal) and the template information, and (1) pth The p-th complex amplitude is determined so that the power of the residual signal obtained by subtracting the p-th reflected sound expressed by multiplying the template by the p-th complex amplitude from the observed signal is minimized. The power of the residual signal obtained by subtracting the p-th reflected sound represented by multiplying the p-th complex amplitude by the p-th template from the observed signal is obtained for each p, and the minimum power is given among them. Determine the template
(2) A function simulating a transfer characteristic for each frequency between an arbitrary position in the space and each of the microphones in the vicinity of the direction D determined by the position corresponding to the determined template (hereinafter referred to as a transfer characteristic function). ) Multiplied by the complex amplitude is subtracted from the observed signal to correct the direction D so that the power of the residual signal E is minimized, and the reflected sound information estimation unit estimates the arrival direction of the reflected sound. The reflected sound information estimation apparatus including. A template representing transfer characteristics for each frequency between the p-th position and each position where M microphones are arranged, where P is a predetermined integer of 2 or more and p is an integer of 1 to P. A storage unit for storing template information that is a set;
A signal obtained by collecting audio signals with M microphones and obtained by converting the M sound collection signals into the frequency domain (hereinafter referred to as an observation signal) and the template information, and (1) pth The p-th complex amplitude is determined so that the power of the residual signal obtained by subtracting the p-th reflected sound expressed by multiplying the template by the p-th complex amplitude from the observed signal is minimized. The power of the residual signal obtained by subtracting the p-th reflected sound represented by multiplying the p-th complex amplitude by the p-th template from the observed signal is obtained for each p, and the minimum power is given among them. Determine the template
(2) A function simulating a transfer characteristic for each frequency between an arbitrary position in the space and each of the microphones in the vicinity of the direction D determined by the position corresponding to the determined template (hereinafter referred to as a transfer characteristic function). ) Is multiplied by the complex amplitude and subtracted from the observed signal to estimate the arrival direction of the reflected sound by correcting the direction D so that the power of the residual signal E is minimized. A reflected sound information estimation device including a reflected sound information estimation unit that estimates the complex amplitude multiplied by the transfer characteristic function corresponding to as the arrival amplitude of the reflected sound. A template representing a transfer characteristic for each frequency between the p-th position and each position where M microphones are arranged, where P is a predetermined integer of 2 or more and p is an integer satisfying 1 ≦ p ≦ P. A storage unit for storing template information that is a set of
A sound signal obtained by picking up sound signals with M microphones is input to a signal (hereinafter referred to as an observation signal) obtained by converting M sound pickup signals into the frequency domain and the template information, and Q is set to 1 or more. A predetermined integer, q is an integer between 1 and Q, and for each q, (1) the pth reflected sound represented by multiplying the pth template by the pth complex amplitude is the qth minimum P-th complex amplitude is determined so that the power of the residual signal obtained by subtracting from the residual signal (where the first minimum residual signal is the observed signal) is minimized. The power of the q + 1th residual signal obtained by subtracting the pth reflected sound represented by multiplying the pth complex amplitude by the pth template from the qth minimum residual signal is obtained for each p, and The te that gave the least power Plate to determine,
(2) A function simulating a transfer characteristic for each frequency between an arbitrary position in the space and each of the microphones in the vicinity of the direction D determined by the position corresponding to the determined template (hereinafter referred to as a transfer characteristic function). ) Multiplied by the complex amplitude is subtracted from the q-th minimum residual signal, and the direction D of the reflected sound is estimated by correcting the direction D so that the power of the residual signal E is minimized. A reflected sound information estimation device including a reflected sound information estimation unit. A template representing a transfer characteristic for each frequency between the p-th position and each position where M microphones are arranged, where P is a predetermined integer of 2 or more and p is an integer satisfying 1 ≦ p ≦ P. A storage unit for storing template information that is a set of
A sound signal obtained by picking up sound signals with M microphones is input to a signal (hereinafter referred to as an observation signal) obtained by converting M sound pickup signals into the frequency domain and the template information, and Q is set to 1 or more. A predetermined integer, q is an integer between 1 and Q, and for each q, (1) the pth reflected sound represented by multiplying the pth template by the pth complex amplitude is the qth minimum P-th complex amplitude is determined so that the power of the residual signal obtained by subtracting from the residual signal (where the first minimum residual signal is the observed signal) is minimized. The power of the q + 1th residual signal obtained by subtracting the pth reflected sound represented by multiplying the pth complex amplitude by the pth template from the qth minimum residual signal is obtained for each p, and The te that gave the least power Plate to determine,
(2) A function simulating a transfer characteristic for each frequency between an arbitrary position in the space and each of the microphones in the vicinity of the direction D determined by the position corresponding to the determined template (hereinafter referred to as a transfer characteristic function). ) Multiplied by the complex amplitude is subtracted from the q-th minimum residual signal, and the direction D of the reflected sound is estimated by correcting the direction D so that the power of the residual signal E is minimized. And a reflected sound information estimation unit including a reflected sound information estimation unit that estimates the complex amplitude multiplied by the transfer characteristic function corresponding to the arrival direction as the arrival amplitude of the reflected sound. In the reflected sound information estimation device according to any one of claims 1 to 4,
The power of the residual signal is the power obtained by adding over all the frequencies,
The reflected sound information is characterized in that the arrival direction is estimated by correcting the direction D so that the power of the residual signal E obtained by adding over all the frequencies is minimized. Estimating device. In the reflected sound information estimation device according to any one of claims 1 to 5,
Ω is the frequency, Ω is the set of frequencies ω, i is the imaginary unit, c is the speed of sound, the p th position [x _p , y _p , z _p ] and the m th (1 ≦ m ≦ M) microphone are arranged. S _pm (ω), the transfer characteristic between positions [u _m , v _m , w _m ], where
As, the template _{S p (ω) = {S} p1 (ω), ..., S pM (ω)} the template information which is a set of _{(ω∈Ω) {S 1 (ω} ), ..., S P (ω )} Further includes a template generation unit for generating (ω∈Ω). In the reflected sound information estimation device according to any one of claims 1 to 6,
The transfer characteristic function is a transfer characteristic for each frequency between an arbitrary position [x, y, z] in space and each position [u _m , v _m , w _m ] where M microphones are arranged. R _m (ω) (1 ≦ m ≦ M), and the transfer characteristic R _m (ω) has a frequency as ω, i as an imaginary unit, and c as a speed of sound.
The reflected sound information estimation apparatus characterized by the above-mentioned. Assuming that P is a predetermined integer of 2 or more, p is an integer of 1 to P, and the storage unit transmits information for each frequency between the pth position and each position where M microphones are arranged. Assume that template information that is a set of templates representing characteristics is stored,
(1) The p-th signal is obtained by using a signal (hereinafter referred to as an observation signal) obtained by collecting M sound pickup signals obtained by picking up sound signals with M microphones into the frequency domain and the template information. The p-th complex amplitude is determined so that the power of the residual signal obtained by subtracting the p-th reflected sound represented by multiplying the template by the p-th complex amplitude from the observed signal is minimized. The power of the residual signal obtained by subtracting the p-th reflected sound represented by multiplying the p-th complex amplitude by the p-th template from the observed signal was obtained for each p, and the minimum power was given among them. Determine the template,
(2) A function simulating a transfer characteristic for each frequency between an arbitrary position in the space and each of the microphones in the vicinity of the direction D determined by the position corresponding to the determined template (hereinafter referred to as a transfer characteristic function). ) Multiplied by complex amplitude is subtracted from the observed signal to correct the direction D so that the power of the residual signal E is minimized, thereby estimating the arrival direction of the reflected sound information. A method for estimating reflected sound information. Assuming that P is a predetermined integer of 2 or more, p is an integer of 1 to P, and the storage unit transmits information for each frequency between the pth position and each position where M microphones are arranged. Assume that template information that is a set of templates representing characteristics is stored,
(1) The p-th signal is obtained by using a signal (hereinafter referred to as an observation signal) obtained by collecting M sound pickup signals obtained by picking up sound signals with M microphones into the frequency domain and the template information. The p-th complex amplitude is determined so that the power of the residual signal obtained by subtracting the p-th reflected sound represented by multiplying the template by the p-th complex amplitude from the observed signal is minimized. The power of the residual signal obtained by subtracting the p-th reflected sound represented by multiplying the p-th complex amplitude by the p-th template from the observed signal was obtained for each p, and the minimum power was given among them. Determine the template,
(2) A function simulating a transfer characteristic for each frequency between an arbitrary position in the space and each of the microphones in the vicinity of the direction D determined by the position corresponding to the determined template (hereinafter referred to as a transfer characteristic function). ) Is multiplied by the complex amplitude and subtracted from the observed signal to estimate the arrival direction of the reflected sound by correcting the direction D so that the power of the residual signal E is minimized. A reflected sound information estimation method including a reflected sound information estimation process in which the complex amplitude multiplied by the transfer characteristic function corresponding to is estimated as an arrival amplitude of the reflected sound. Assuming that P is a predetermined integer equal to or greater than 2, and p is an integer satisfying 1 ≦ p ≦ P, the storage unit is configured for each frequency between the p-th position and each position where M microphones are arranged. Assume that template information that is a set of templates representing transfer characteristics is stored,
Using a signal (hereinafter referred to as an observation signal) obtained by collecting M sound pickup signals obtained by collecting sound signals with M microphones into the frequency domain and the template information, Q is set to 2 or more in advance. For each q, a defined integer, q is an integer between 1 and Q, and (1) the pth reflected sound represented by multiplying the pth template by the pth complex amplitude is the qth smallest The p-th complex amplitude is determined so that the power of the residual signal obtained by subtracting from the residual signal (where the first minimum residual signal is the observed signal) is minimized, and the determined p The power of the q + 1th residual signal obtained by subtracting the pth reflected sound represented by multiplying the pth template by the pth template and the qth residual signal is obtained for each p. The balance that gave the least power To determine the over door,
(2) A function simulating a transfer characteristic for each frequency between an arbitrary position in the space and each of the microphones in the vicinity of the direction D determined by the position corresponding to the determined template (hereinafter referred to as a transfer characteristic function). ) Multiplied by the complex amplitude is subtracted from the q-th minimum residual signal, and the direction D of the reflected sound is estimated by correcting the direction D so that the power of the residual signal E is minimized. A reflected sound information estimation method including a reflected sound information estimation process. Assuming that P is a predetermined integer equal to or greater than 2, and p is an integer satisfying 1 ≦ p ≦ P, the storage unit is configured for each frequency between the p-th position and each position where M microphones are arranged. Assume that template information that is a set of templates representing transfer characteristics is stored,
Using a signal (hereinafter referred to as an observation signal) obtained by collecting M sound pickup signals obtained by collecting sound signals with M microphones into the frequency domain and the template information, Q is set to 2 or more in advance. For each q, a defined integer, q is an integer between 1 and Q, and (1) the pth reflected sound represented by multiplying the pth template by the pth complex amplitude is the qth smallest The p-th complex amplitude is determined so that the power of the residual signal obtained by subtracting from the residual signal (where the first minimum residual signal is the observed signal) is minimized, and the determined p The power of the q + 1th residual signal obtained by subtracting the pth reflected sound represented by multiplying the pth template by the pth template and the qth residual signal is obtained for each p. The balance that gave the least power To determine the over door,
(2) A function simulating a transfer characteristic for each frequency between an arbitrary position in the space and each of the microphones in the vicinity of the direction D determined by the position corresponding to the determined template (hereinafter referred to as a transfer characteristic function). ) Multiplied by the complex amplitude is subtracted from the q-th minimum residual signal, and the direction D of the reflected sound is estimated by correcting the direction D so that the power of the residual signal E is minimized. And a reflected sound information estimation method including a reflected sound information estimation step of estimating the complex amplitude multiplied by the transfer characteristic function corresponding to the arrival direction as an arrival amplitude of the reflected sound. The reflected sound information estimation method according to any one of claims 8 to 11,
The power of the residual signal is the power obtained by adding over all the frequencies,
The reflected sound information is characterized in that the arrival direction is estimated by correcting the direction D so that the power of the residual signal E obtained by adding over all the frequencies is minimized. Estimation method. In the reflected sound information estimation method according to any one of claims 8 to 12,
Ω is the frequency, Ω is the set of frequencies ω, i is the imaginary unit, c is the speed of sound, the p th position [x _p , y _p , z _p ] and the m th (1 ≦ m ≦ M) microphone are arranged. S _pm (ω), the transfer characteristic between positions [u _m , v _m , w _m ], where
As, the template _{S p (ω) = {S} p1 (ω), ..., S pM (ω)} the template information which is a set of _{(ω∈Ω) {S 1 (ω} ), ..., S P (ω )} (Ω∈Ω) is further included in the template generation process, and the reflected sound information estimation method is characterized. In the reflected sound information estimation method according to any one of claims 8 to 13,
The transfer characteristic function is a transfer characteristic for each frequency between an arbitrary position [x, y, z] in space and each position [u _m , v _m , w _m ] where M microphones are arranged. R _m (ω) (1 ≦ m ≦ M), and the transfer characteristic R _m (ω) has a frequency as ω, i as an imaginary unit, and c as a speed of sound.
The reflected sound information estimation method characterized by the above-mentioned.       The program for making a computer perform the process of the reflected sound information estimation method in any one of Claims 8-14.