JP5216056B2 - Reflected sound information estimation apparatus, reflected sound information estimation method, program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technology for estimating reflection sound information from a sound pickup signal. <P>SOLUTION: As for a reflection sound expressed by multiplying a function (transmission characteristic function) simulating the transmission characteristics of every frequency between an arbitrary position in a space and each microphone by a complex amplitude by using an observation signal obtained by converting each of M pieces of sound pickup signals obtained by performing the sound pickup of a voice signal by M pieces of microphones into a frequency area, directions and Q pieces of complex amplitudes to be determined according to each position in a space corresponding to Q pieces of transmission characteristic functions are estimated in a batch so that the power of a residual signal to be obtained by subtracting Q pieces of reflection sounds corresponding to different Q pieces of positions from the observation signal can be minimized, and the estimated Q pieces of directions are defined as the arrival directions of the Q pieces of reflection sounds, and the estimated Q pieces of complex amplitudes are defined as the arrival amplitudes of the Q pieces of reflection sounds. <P>COPYRIGHT: (C)2012,JPO&amp;INPIT

Description

  The present invention relates to a technique for estimating information (arrival amplitude, arrival direction) relating to reflected sound from a collected sound signal obtained by collecting a sound signal with a microphone.

  A system for exchanging voice information such as telephone calls and voice conferences is generally called a voice communication system. In a voice communication system, it is very important to obtain information about the reflected sound (arrival amplitude, arrival direction, etc.). In a reverberant environment such as a conference room, the collected sound signal collected through the microphone is reflected not only from the direct sound coming from the sound source such as the speaker but also from the floor, wall or ceiling. Reflected sound is mixed. Therefore, when a speaker's utterance is recorded in such a reverberant environment, the reflected sound is mixed with a delay from the direct sound, making it difficult to hear. If arrival information of each reflected sound can be estimated from the collected sound signal and the reflected sound can be removed, it is possible to recover a sound that is easy to hear. Here, Non-Patent Document 1 is given as a conventional study for estimating reflected sound information.

  A functional configuration for realizing the technology disclosed in Non-Patent Document 1 is shown in FIG. The processing procedure in this technique is as follows.

1. The sound source signal radiated from the impulse sound source 100 is picked up using 4ch microphones 110-1, 110-2, 110-3, 110-4. The AD converter 120 converts the collected analog signal into a digital signal x ^→ (t) = [x ₁ (t), x ₂ (t), x ₃ (t), x ₄ (t)] ^T . Here, [•] ^T represents transposition. t represents a discrete time index. Assume that four microphones are arranged at the apexes of a regular tetrahedron.

2. The impulse response calculation unit 130 receives the digital signal x ^→ (t) = [x ₁ (t), x ₂ (t), x ₃ (t), x ₄ (t)] ^T as an input, and the impulse response of each microphone. h ^→ (t) = [h ₁ (t), h ₂ (t), h ₃ (t), h ₄ (t)] ^T is calculated. The impulse response calculation method includes a TSP method, an M-sequence method, and the like, and any method may be used to calculate the impulse response.

3. The virtual sound source calculation unit 140 receives 4ch impulse response h ^→ (t) = [h ₁ (t), h ₂ (t), h ₃ (t), h ₄ (t)] ^T as input, and generates virtual sound source information. v ^→ = [v ^→ ₁ ,…, v ^→ _D ] ^T is output. D represents the number of virtual sound sources. A virtual sound source is a sound source that is virtually present to represent the arrival amplitude, arrival direction, and arrival time of each reflected sound. The virtual sound source will be described with reference to FIG. FIG. 2 shows a path for receiving a sound source signal reflected by the right wall with a microphone. The sound source signal reflected from the right wall (reflected sound) is equivalent to the signal coming directly from the position written as “virtual sound source” (however, the effects of attenuation and distance attenuation due to reflection on the wall are not receive).

Details of this prior art will be described. When the impulse response is measured at four adjacent sound receiving points (microphone positions), there is a slight difference in the arrival time of the reflected sound. By using the cross-correlation of the short section of the impulse response and associating the reflected sound with each microphone, as shown in FIG. 3, the arrival time t _1n at each sound receiving point regarding the nth reflected wave, t _2n , t _3n , t _4n (1 ≦ n ≦ D) are obtained. Each virtual sound source information v _n ^→ = [X _n , Y _n , Z _n , S _n ] ^T is obtained when the length of the side of the regular tetrahedral microphone array is d and the speed of sound is c. Here, X _n , Y _n , and Z _n represent the position of the n-th virtual sound source (see equations (1) to (3)), which has information corresponding to the arrival direction and arrival time of each reflected sound. . Further, S _n represents the strength of the n-th virtual source, determined by the average of the amplitudes of the an n-th reflected sound associated with impulse 4ch.

Yoshio Yamazaki et al., “Spatial information obtained from impulse responses of four adjacent points”, Proc. Of the Acoustical Society of Japan, 1981, pp.759-760.

  According to the prior art, in order to estimate the “arrival amplitude”, “arrival direction”, and “arrival time” of the reflected sound called virtual sound source information, it is necessary to prepare an impulse response in advance. However, since it is necessary to observe using a special signal in order to prepare an impulse response, the condition that impulse responses at all positions are prepared in advance is not realistic.

  Therefore, an object of the present invention is to provide a technique for estimating reflected sound information ("arrival direction" or "arrival amplitude" of reflected sound) from a collected sound signal without using a special signal.

M in each of P directions (where P is a predetermined integer of 2 or more) viewed from a microphone array composed of M microphones (where M is an integer of 4 or more). For each of the microphones, a function that simulates the transfer characteristics for each frequency (hereinafter referred to as transfer characteristics function) is prepared in advance as a template, and is emitted from a single sound source in a plurality of directions (hereinafter referred to as voice arrival directions). Q) using a signal (hereinafter referred to as an observation signal) obtained by collecting M sound signals obtained by collecting sound signals arriving from M microphones by using M microphones, and converting Q into two or more in advance. As a defined integer, Q-1 reflected sounds that are different from one direct sound and direct sound (hereinafter, Q-1 reflected sounds that are different from one direct sound and direct sound are combined. Q speech signals) are each expressed by multiplying the transfer characteristic function for each frequency for the M microphones in the direction of speech arrival of the Q speech signals viewed from the microphone array by the complex amplitude. The Q voice arrival directions specified by the Q templates selected from the templates are set as the initial directions of the Q voice signals, and the complex amplitude corresponding to each initial direction is determined as the initial amplitude. Starting from the Q initial directions and the Q initial amplitudes, the power of the residual signal obtained by subtracting the Q speech signals corresponding to the different Q speech arrival directions from the observed signal becomes small. Thus, Q speech arrival directions and Q complex amplitudes corresponding to Q speech signals are collectively obtained by correcting Q speech arrival directions and Q complex amplitudes as described above. A constant.

  According to the present invention, in the space corresponding to the Q transfer characteristic functions, the power of the residual signal obtained by subtracting the Q reflected sounds corresponding to the different Q positions from the observation signal is minimized. Since the direction determined by each position and the Q complex amplitudes are collectively estimated, the reflected sound information is estimated from the collected sound signal without using a special signal for the sound source signal to obtain the impulse response. It is possible. When the reflected sound information is obtained, it can be applied to applications such as estimation of the sound source direction and voice enhancement (collection of far-field sounds and sound collection by distance) that could not be realized by conventional voice information processing technology.

The figure which shows the function structure of the reflected sound information estimation technique in a prior art. The figure for demonstrating a virtual sound source. The figure for demonstrating matching of the reflected sound in a prior art. The figure which shows the function structure of the reflected sound information estimation apparatus which concerns on 1st Embodiment. The figure which shows the process sequence of the reflected sound information estimation method which concerns on 1st Embodiment. The figure which shows the structural example of a two-dimensional microphone array. p-th point _{[x p, y p, z} p] diagram for explaining the transfer characteristic between the m-th receiving point _{[u m, v m, w} m]. The figure for demonstrating that the sound pressure distribution in a certain plane observed using the microphone array shown in FIG. 6 is obtained by superimposing the direct sound, the reflected sound 1, and the reflected sound 2, for example. (A) The figure for demonstrating that only the information regarding an estimated arrival direction should be extracted ideally. (B) The figure for demonstrating that the information regarding directions other than an estimated arrival direction will actually be mixed. The figure for demonstrating reducing the influence of directions other than an estimated arrival direction by summarizing the power of a residual signal over all the frequencies. The figure which shows the sound pressure distribution at the time of using a two-dimensional matrix microphone array of a practical use level, and its decomposition | disassembly.

<< First Embodiment >>
In the present invention, a voice signal (sound source signal) radiated from a sound source such as an utterance signal is collected by a microphone array composed of a plurality of microphones (sound collection signal), or “arrival direction” of reflected sound, Estimate the “arrival amplitude” and “arrival direction”. The functional configuration and processing flow of the first embodiment are shown in FIGS.

The sound source signal radiated from the sound source 200 is collected using the Mch microphones 210-1,..., 210-M (step S1). A value greater than 4 is desirable for M. The AD converter 220 converts the collected analog signal into a digital signal xx ^→ (t) = [xx ₁ (t),..., Xx _M (t)] ^T (step S2). Here, [•] ^T represents transposition. t represents a discrete time index.

It is desirable to arrange the M microphones at equal intervals in two dimensions or three dimensions. This is to uniquely determine the correspondence between the arrival direction of the reflected sound and the template (which will be described later, which simulates the transfer characteristic of the reflected sound). In principle, the present invention can be implemented even if the microphones are arranged one-dimensionally or not at regular intervals, but there is a one-to-one relationship between the transmission characteristics of the reflected sound and the arrival directions of the reflected sound. Therefore, it is desirable to arrange them at equal intervals in two dimensions or three dimensions. An example when microphones are arranged at equal intervals on a two-dimensional plane is shown in FIG. The microphone interval d is preferably set so as to satisfy the spatial sampling theorem. When the spatial sampling theorem is satisfied, the microphone interval d is a numerical value that satisfies Equation (4). c is the speed of sound, and f is the frequency to be analyzed. For example, when analyzing a frequency of 4 kHz, it is preferable to set the microphone interval to about 4 cm.

Frame division unit 230, a digital signal xx ^→ AD conversion unit 200 is output _{(t) = [xx 1 (} t), ..., xx M (t)] as input ^T, a digital signal consisting of a plurality of samples for each channel The signal x ^→ (k) = [x ₁ (k),..., X _M (k)] ^T divided into groups (frames) is output (step S3). k is an index representing a frame number. The frame division is a process of buffering and outputting W points for each digital signal xx _i (t) (1 ≦ i ≦ M) of each channel. W depends on the sampling frequency, but in the case of 16 kHz sampling, around 512 points are appropriate.

The frequency domain converter 240 receives the digital signal x ^→ (k) of each frame as an input, and the frequency domain signal X ^→ (ω, k) = [X ₁ (ω, k),..., X _M (ω, k )] Converted to ^T and output (step S4). This signal X ^→ (ω, k) is called an observation signal. Here, ω indicates a discrete frequency index (there is a relationship of ω = 2πf between the frequency f and the angular frequency ω, and therefore the frequency index ω may be identified with the angular frequency ω. Hereinafter, with respect to ω, “frequency index” is also simply referred to as “frequency”), and k indicates a frame index. One method for transforming into the frequency domain is discrete Fourier transform, but other methods may be used as long as the transform is performed into the frequency domain. The observation signal X ^→ (ω, k) in the frequency domain is output for each frequency ω and frame k.

The template generation unit 250 includes template information S ^→ (ω) = [S ₁ ^→ (ω),..., Which is a set of P templates _Sp ^→ (ω) (however, for convenience of calculation, expressed in vector). , S _P ^→ (ω)] ( ^∀ ω∈Ω; Ω is a set of frequency indices ω) is generated for each frequency ω (step Sp). This process is usually performed prior to each process of steps S1-S4. P represents the total number of templates, and is set in advance to an integer value of 2 or more. The larger the total number of templates P, the more accurately the reflected sound information is estimated. However, since the amount of calculation increases, for example, P = 1000 is preferable. This processing is performed in advance before observing the signal with the microphone. Further, unless the position of the microphone (for example, the distance d between the microphones) is changed or the total number P of templates is not changed, it is usually unnecessary to recreate the template every time. The “template” referred to here is a simulation of a transfer characteristic (acoustic propagation characteristic) corresponding to the arrival direction of the reflected sound. The p-th (1 ≦ p ≦ P) template S _p ^→ (ω) = [S _p1 (ω),..., S _pM (ω)] ^T (ω∈Ω) is a predetermined p-th point [ x _p , y _p , z _p ] and M sound receiving points (where the sound receiving point is the position where the microphone is placed, and the m th (1 ≦ m ≦ M) sound receiving point is represented by [u _m , v _m , w _m ]) (see FIG. 7). An example of a calculation formula for each element S _pm (ω) of the p-th template _Sp ^→ (ω) is shown in Formula (5). The symbol i represents an imaginary unit.

Direction information θ _p ^→ (ω) is associated with the p-th template S _p ^→ (ω). The direction information θ _p ^→ (ω) is a three-dimensional orthogonal coordinate system that serves as a reference for the position coordinates of the p th point [x _p , y _p , z _p ] and the sound receiving point [u _m , v _m , w _m ]. The direction of viewing the p-th point [x _p , y _p , z _p ] from the origin of, for example, two declinations in a spherical coordinate system (having a common origin with the origin of the 3D Cartesian coordinate system) ( _Polar angle θ _{p, pol} and azimuth angle θ _{p, azi} ). That is, θ _p ^→ (ω) = [θ _{p, pol} (ω), θ _{p, azi} (ω)]. If the p-th template S _p ^→ (ω) is associated with the p-th point [x _p , y _p , z _p ], the direction information θ _p ^→ (ω) is the position [x _p , y _p , since can be calculated from z _p], p-th template S _p ^→ (ω) be the direction information θ _p ^→ (ω) is associated with is not a requirement. Since the three-dimensional orthogonal coordinate system and the spherical coordinate system can be converted to each other (coordinate conversion), the right side of Equation (5) is not the position [x, y, z] but the direction information θ _p ^→ (ω). = [Θ _{p, pol} (ω), θ _{p, azi} (ω)], for example, can also be expressed as shown in equation (5a). Here, d is a microphone interval, the microphone array is a two-dimensional microphone array of Φ rows and Ξ columns (Φ × Ξ = M), and the position of the mth microphone is φ rows and ξ columns (1 ≦ φ ≦ Φ, 1 ≦ ξ ≦ Ξ).

Further, when the template corresponds to the direction as in the first embodiment, the positions of the P points [x _p , y _p , z _p ] (1 ≦ p ≦ P) are positions having different directions from each other. For example, assuming that each point [x _p , y _p , z _p ] is at an equal distance sufficiently away from the origin, different P points on the sphere centered on the origin may be used. The reason why each point [x _p , y _p , z _p ] is located sufficiently away from the origin is that the signal radiated from the sound source or virtual sound source is transmitted spherically but sufficiently separated from the sound source or virtual sound source This is because a direct sound or reflected sound can be simulated as a plane wave in a local region at the position (origin). However, this does not mean that the template information includes a template corresponding to a position in the same direction. It is assumed that the microphone array is disposed in the vicinity (local region) of the origin of the coordinate system.

The template storage unit 260 stores the template information S ^→ (ω) output from the template generation unit 250 and plays a role of providing the template information S ^→ (ω) to the reflected sound information estimation unit 270 at the time of analysis.

The reflected sound information estimation unit 270 receives a frequency domain observation signal X ^→ (ω, k) and template information S ^→ (ω) as an input, and a set of Q reflected sound information components rs _q ^→ (ω, k) ( However, the reflected sound information rs ^→ (ω, k) = [rs ₁ ^→ (ω, k),..., Rs _Q ^→ (ω, k)] ^T is calculated for each frame. k is output for each frequency ω (step S5). Here, Q represents the total number of reflected sounds to be estimated, and is set in advance to an integer value of 1 or more. The q-th (1 ≦ q ≦ Q) reflected sound information component rs _q ^→ (ω, k) is expressed as rs _q ^→ (ω, k) = [rsA _q (ω, k), rsB _q (ω, k)] RsA _q (ω, k) is the arrival amplitude of the qth reflected sound, and rsB _q (ω, k) is the arrival direction of the qth reflected sound.

  The principle of estimating the reflected sound information will be described. An example of the sound pressure distribution in a certain plane observed using a two-dimensional microphone array as shown in FIG. 6 is shown as a shading diagram at the left end of FIG. Regarding the view of the sound pressure distribution shown as a shading diagram, the black portion indicates that the sound pressure is low and the white portion indicates that the sound pressure is high. The observed sound pressure distribution includes not only the sound pressure distribution of the direct sound but also the sound pressure distribution of the reflected sound. When the direct sound and the reflected sound come sufficiently far away, each sound pressure distribution on the two-dimensional plane has a striped pattern as shown in the three shades on the right side of FIG. Striped “shading” corresponds to the arrival amplitude of direct sound or reflected sound, and “rotation / period” corresponds to the direction of arrival of direct sound or reflected sound. In the example of FIG. 8, it is shown that the sound pressure distribution of the observation signal is configured by superimposing the sound pressure distributions of the direct sound, the reflected sound 1 and the reflected sound 2 having different arrival amplitudes and directions. When considered in the frequency domain, the direct sound and each reflected sound are represented by a complex sine wave whose frequency changes according to the direction of arrival, and the observation signal is a superposition of the direct sound and multiple complex sine waves corresponding to each reflected sound. Represented as: Incidentally, the problem to be solved by the present invention is to estimate the arrival amplitude and / or the arrival direction of the reflected sound using only the observation signal. The solution to this problem is to estimate the “tone” and “rotation / cycle” of the striped pattern corresponding to the direct sound and the reflected sounds of the three shades on the right side of FIG. 8 from the sound pressure distribution drawn at the left end of FIG. Corresponding to.

An outline of a method for estimating the reflected sound information rs ^→ (ω, k) will be described. In this embodiment, Q reflected sound information components rs _q ^→ (ω, k) (1 ≦ q ≦ Q) are estimated in a lump rather than sequentially. When the reflected sound information components are estimated in order, estimation errors are accumulated sequentially, which may adversely affect the accuracy of the arrival direction and arrival amplitude due to the subsequent estimation. Can be eliminated. In the example of FIG. 8, Q = 3, and the direct sound and the two reflected sounds represented by the three shades on the right side of FIG. 8 are collectively estimated from the observation signal. This estimation is performed so that the power of the residual signal obtained by removing Q reflected sounds from the observed signal is minimized. In this sense, since the distinction between direct sound and reflected sound only depends on the difference in power, understanding the reflected sound with the strongest power included in the observed signal as “direct sound”, the following explanation Then, it is assumed that direct sound and reflected sound are handled without distinction. Note that Q is preferably set to about 30 although it depends on the calculation power and the application using the reflected sound information.

Note that the sound pressure distribution in FIG. 8 is shown as a high-resolution shading map, but in order to show the sound pressure distribution as such a high-resolution shading chart, an extremely large number of microphones are required, which is not practical. Absent. On the other hand, even when, for example, 100 microphones are used as a 10 × 10 two-dimensional matrix microphone array as a practical two-dimensional matrix microphone array, the sound pressure shown as a rough (low resolution) gray scale diagram (see FIG. 11). Only a distribution can be obtained. Thus, from a practical point of view, it is required to accurately estimate the arrival amplitude and direction of the reflected sound in a situation where only a low-resolution sound pressure distribution can be obtained. In the present invention, a plane wave arriving from an arbitrary position is specifically expressed in order to improve spatial resolution (formulation), and reflected sound with low power can be estimated under the influence of reflected sound with high power. In order to prevent disappearance, the already estimated reflected sound is removed from the observation signal to estimate the next reflected sound (decomposition). The formulation is as described in the template information, and the decomposition is as described in the outline of the estimation method of the reflected sound information rs ^→ (ω, k).

The method for estimating the reflected sound information rs ^→ (ω, k) described above will be described in detail. Prior to explanation, symbols are defined. E ^→ (ω, k) = [E ₁ (ω, k),…, E _M (ω, k)] ^T , q th The reflected sound (representing direct sound when q = 1) is defined as A _q (ω, k) R _q ^→ (ω, θ _q ^→ (ω, k)). R _q ^→ (ω, θ _q ^→ (ω, k)) = (R ₁ (ω, θ _q ^→ (ω, k)), ..., R _M (ω, θ _q ^→ (ω, k))] ^T is a function that simulates the transfer characteristics for each frequency between an arbitrary position [x, y, z] in space and each microphone (hereinafter referred to as the transfer characteristic function). Any function that simulates the transfer characteristics may be used. The reason why such a transfer characteristic function is a component of reflected sound is that Q templates corresponding to Q directions considered to be closest to each of the Q arrival directions to be estimated of Q reflected sounds. This is for improving the accuracy of estimating the arrival direction of the reflected sound by correcting Q directions D in the vicinity of the directions D corresponding to the Q templates. The optimization of the sound A _q (ω, k) R _q ^→ (ω, θ _q ^→ (ω, k)) will be described later). Usually, each transfer characteristic R _m (ω, θ _q ^→ (ω, k)) constituting the transfer characteristic function and the calculation formula for each element S _pm (ω) of the template are the same. In this case, between the position [x, y, z] in the direction represented by the direction information θ _q ^→ (ω, k) and the m th sound receiving point [u _m , v _m , w _m ] The transfer characteristic R _m (ω, θ _q ^→ (ω, k)) for each frequency is expressed by equation (6). Note that the position [x, y, z] in the direction represented by the direction information θ _q ^→ (ω, k) may be, for example, a position on a spherical surface sufficiently away from the coordinate system origin. The reason why the position [x, y, z] is sufficiently distant from the origin is as described above. Specifically, the position [x, y, z] is a sound source in the local region where the microphone array is arranged. Or it is preferably an arbitrary position in a space at a distance that can simulate a direct sound or reflected sound from a virtual sound source as a plane wave. Since the three-dimensional orthogonal coordinate system and the spherical coordinate system can be converted to each other (coordinate conversion), the right side of Equation (6) is not the position [x, y, z] but the direction information θ _q ^→ (ω, Using k) = [θ _{q, pol} (ω, k), θ _{q, azi} (ω, k)], for example, it can also be expressed as in equation (6a). Here, d is a microphone interval, the microphone array is a two-dimensional microphone array of Φ rows and Ξ columns (Φ × Ξ = M), and the position of the mth microphone is φ rows and ξ columns (1 ≦ φ ≦ Φ, 1 ≦ ξ ≦ Ξ).

A _q (ω, k) constituting the reflected sound is a template R _q ^→ (ω, θ _q ^→ (ω, k)) such as the phase of the sound source 200 itself, reflection on the wall, attenuation due to distance, and reflected sound. This corresponds to the arrival amplitude. When the above-described method for estimating the Q reflected sound information components at once is expressed by an equation, the equation (7) is obtained. However, 1 ≦ q ≦ Q.

Next, a method for collectively optimizing Q reflected sounds A _q (ω, k) R _q ^→ (ω, θ _q ^→ (ω, k)) will be described.
Q optimized reflected sounds A _q (ω, k) R _q ^→ (ω, θ _q ^→ (ω, k)) are residual signals obtained by removing Q reflected sounds from the observed signal. E ^→ (ω, k) power of ^{(E → (ω, k)} ) H E → (ω, k) is determined according to the criteria to minimize. Specifically, if it is noted that the transfer characteristic function R _q ^→ (ω, θ _q ^→ (ω, k)) is determined by the direction information θ _q ^→ (ω, k), Q reflected sounds A _q (ω, k) R _q ^→ (ω, θ _q ^→ (ω, k)) (1 ≦ q ≦ Q) Each parameter A _q (ω, k), θ _q ^→ (ω, k) (1 The optimum values A _{q, opt} (ω, k), θ _{q, opt} ^→ (ω, k) (1 ≦ q ≦ Q) of ≦ q ≦ Q are obtained by the equation (8). Note that the symbol H represents conjugate transposition. In equation (8), {(A _{q, opt} (ω, k), θ _{q, opt} ^→ (ω, k))} _{q∈ {1,..., Q}} is {(A _{1, opt} (ω , k), θ _{1, opt} ^→ (ω, k)),…, (A _{q, opt} (ω, k), θ _{q, opt} ^→ (ω, k)),…, (A _{Q, opt} (ω , k), θ _{Q, opt} ^→ (ω, k))}, {(A _q (ω, k), θ _q ^→ (ω, k))} _{q∈ {1,.} {(A ₁ (ω, k), θ ₁ ^→ (ω, k)), ..., (A _q (ω, k), θ _q ^→ (ω, k)), ..., (A _Q (ω, k ), Θ _Q ^→ (ω, k))}.

At this time, the q-th reflected sound information component rs _q ^→ (ω, k) = [rsA _q (ω, k), rsB _q (ω, k)] is given by equations (9) and (10).

  Various specific calculation methods of the formula (8) are conceivable, but an example is shown here.

§1.1 Initial value setting of direction information First, initial values θ _{ini, q} ^→ (ω, k) (1 ≦ q ≦ Q) of Q pieces of direction information θ _q ^→ (ω, k) are set. As a method for determining Q initial values θ _{ini, q} ^→ (ω, k) (1 ≦ q ≦ Q), in this embodiment, initial values θ _{ini of} Q direction information θ _q ^→ (ω, k). _{, q} ^→ (ω, k) (1 ≦ q ≦ Q) will be described with reference to the observation signal X ^→ (ω, k) and the template information S ^→ (ω). According to this method, Q templates corresponding to Q directions considered to be closest to each of Q estimated directions of arrival are determined, and direction information corresponding to the determined Q templates is determined. The initial value θ _{ini, q} ^→ (ω, k) of Q pieces of direction information θ _q ^→ (ω, k) may be set to 1 ≦ q ≦ Q. In this case, there is a relationship of Q <P between P and Q.

Therefore, in order to determine the template as described above from the template information, for convenience, the q-th reflected sound is expressed as A _q (ω, k, g (ω, q)) S _{g (ω, q)} ^→ (ω ). Here, g (ω, q) represents an index of a template that can represent the q-th reflected sound most accurately in the template information. The coefficient A _q (ω, k, g (ω, q)) constituting the reflected sound is determined by the template S _{g (ω, q)} ^→ (ω ) And the reflected sound. In this case, the remaining signal obtained by removing the q-th reflected sound A _q (ω, k, g (ω, q)) S _{g (ω, q)} ^→ (ω) from the observed signal X ^→ (ω, k). The difference signal E _q ^→ (ω, k) is expressed as shown in Equation (11).

The q-th reflected sound A _q (ω, k, g (ω, q)) S _{g (ω, q)} ^→ (ω) is obtained from the residual signal E _q ^→ (ω, k) based on the equation (11). Power (E _q ^→ (ω, k)) Estimated according to a standard that minimizes ^H E _q ^→ (ω, k). There are various estimation methods, but one of them will be described.

The reflected sound consists of two elements, A _q (ω, k, g (ω, q)) and S _{g (ω, q)} ^→ (ω). Is required. First, the p-th template S _p ^→ (ω) minimizes the power (E _q ^→ (ω, k)) ^H E _q ^→ (ω, k) of the residual signal E _q ^→ (ω, k). The coefficient A _q (ω, k, p) in the case where it is assumed that the template is an optimal template is obtained by the equation (12) based on the least square method. It should be noted that at this stage, q on the left side of Equation (12) has no meaning.

Next, among the P coefficients A _q (ω, k, p) (1 ≦ p ≦ P) obtained based on the equation (12), the Q coefficients A _q are ordered in descending order of their absolute values. The index q (1 ≦ q ≦ Q) of (ω, k, p) is determined (see equation (13)). The symbol Λ is a set obtained by subtracting the set of indexes determined by the equation (13) from the entire set {1,..., P,..., P} of the index p, and Λ = {1,. , P}-{g (ω, 1),..., G (ω, q-1)}.

Accordingly, the initial value θ _{ini, q} ^→ (ω, k) (1 ≦ q ≦ Q) of the Q pieces of direction information θ _q ^→ (ω, k) is expressed by Q pieces of g (ω , q) (S ≦ _G ≦ ω ≦ q ≦ Q) (1 ≦ q ≦ Q) ^→ (ω) (1 ≦ q ≦ Q) direction information θ _{g (ω, q)} ^→ (ω) = [θ _{g (ω, q), pol} (ω), θ _{g (ω, q), azi} (ω)] (1 ≦ q ≦ Q). That is, θ _{ini, q} ^→ (ω, k) = [θ _{g (ω, q), pol} (ω), θ _{g (ω, q), azi} (ω)] (1 ≦ q ≦ Q). Note that the initial value θ _{ini, q} ^→ (ω, k) does not depend on the frame index k.

§1.2 coefficients A _q (ω, k) initial value setting Next, Q-number of coefficients A _q (ω, k) initial value A _{ini of, q (ω,} k) is set to. There are various methods for determining the Q initial values A _{ini, q} (ω, k) (1 ≦ q ≦ Q). Here, as an example, the Q initial values A _{ini on the} power minimization basis are considered. _{, q} (ω, k) (1 ≦ q ≦ Q) will be described. First, A _q (ω, k, p) = 0 (1 ≦ p ≦ P). The initial value A _{ini, q} (ω, k) (1 ≦ q ≦ Q) is the power (E ^→ (ω, k)) ^H E ^→ (ω, k) of the residual signal E ^→ (ω, k). ) Based on the method of least squares so as to minimize. In equation (14), F _q ^→ (ω, k) is given by equation (15). In equation (15), Υ = {1, ..., q-1, q + 1, ..., Q}, and F _q ^→ (ω, k) is the residual obtained by removing the q-th reflected sound from the observed signal Signal. Note that Q initial values θ _{ini, q} ^→ (ω, k) determined in §1.1 are used as Q pieces of direction information θ _q ^→ (ω, k). R _q ^→ (ω, θ _q ^→ (ω, k)) used in the equation (14) is obtained from the initial value θ _{ini, q} ^→ (ω, k) of the direction information and the equation (6).

§2 Optimization of reflected sound Next, using the initial value θ _{ini, q} ^→ (ω, k) (1 ≦ q ≦ Q) of Q pieces of direction information θ _q ^→ (ω, k) as a starting point, the equation (7 ) Q reflected sounds A _q (ω to minimize the power (E ^→ (ω, k)) ^H E ^→ (ω, k) of the residual signal E ^→ (ω, k) , k) R _q ^→ (ω, θ _q ^→ (ω, k)) (1 ≦ q ≦ Q) is optimized collectively. Each reflected sound is composed of two elements: coefficient A _q (ω, k) and R _q ^→ (ω, θ _q ^→ (ω, k)). Necessary. There are various optimization methods, but one of them (gradient method) will be described. In the illustrated method, the correction of the direction information θ _q ^→ (ω, k) and the correction of the coefficient A _q (ω, k) are alternately repeated a predetermined number of times (δ times), thereby Q reflected sounds A _q (ω, k) R _q ^→ (ω, θ _q ^→ (ω, k)) (1 ≦ q ≦ Q) is optimized. For example, δ is set to a value of about 100, but may be 1.

§2.1 Correction of direction information Q pieces of direction information θ _q ^→ (ω, k) = [θ _{q, pol} (ω, k), θ _{q, azi} (ω, k)] (1 ≦ q ≦ Q) Is corrected by updating according to the equation (16). When the processing of §2.1 is performed for each q (1 ≦ q ≦ Q) for the first time, the direction information θ _q ^→ (ω, k) on the right side of Expression (16) is the initial value obtained by the processing of §1.1 If the value θ _{ini, q} ^→ (ω, k) and the processing of §2.1 is not the first time, the direction information θ _q ^→ (ω, k) on the right side of equation (16) is the previous §2.1. The direction information obtained by the processing is used. Further, when the processing of §2.1 is performed for the first time, the coefficient A _q (ω, k) (q∈Υ) used for calculating the power (F _q ^→ (ω, k)) ^H F _q ^→ (ω, k) ) Using A _{ini, q} (ω, k) (1 ≦ q ≦ Q) obtained in the equation (14), and the processing of §2.1 is not the first time, the power (F _q ^→ (ω, k )) The coefficient A _q (ω (k, k) (q∈Υ) used in the calculation of ^H F _q ^→ (ω, k) is the coefficient A _q ( ω, k) (1 ≦ q ≦ Q) is used. The step widths α ₁ and α ₂ are small positive constants, which are determined in consideration of the convergence speed and the like, and are each about 0.1, for example.

§2.2 Correction of coefficients The correction of the Q coefficients A _q (ω, k) (1 ≦ q ≦ Q) is based on the least squares method and a new coefficient A _q (ω, k) according to the equation (17). This is performed by obtaining (1 ≦ q ≦ Q). R _q ^→ (ω, θ _q ^→ (ω, k)) used in Expression (17) is obtained from the direction information θ _q ^→ (ω, k) obtained by the processing of §2.1 and Expression (6). . When the processing of §2.2 is performed for the first time as the coefficient A _q (ω, k) (q∈Υ) used in the calculation of F _q ^→ (ω, k) in Expression (17), When the initial value A _{ini, q} (ω, k) obtained in the processing is used and the processing in §2.2 is not the first time, the coefficient A _q (ω, k) obtained in the processing in the previous §2.2 is used. ) (1 ≦ q ≦ Q) is used.

Q combinations (A _q (ω, k), θ _q ^→ ) of coefficient A _q (ω, k) and direction information θ _q ^→ (ω, k) obtained at the end of δ iterations (ω, k)) (1 ≦ q ≦ Q) is {(A _{q, opt} (ω, k), θ _{q, opt} ^→ (ω, k))} _{q∈ {1,..., Q}} Q reflected sound information components rs _q ^→ (ω, k) (1 ≦ q ≦ Q). That is, the q-th reflected sound information component rs _q ^→ (ω, k) = [rsA _q (ω, k), rsB _q (ω, k)] is given by the equations (18) and (19).

Through the above process, Q reflected sound information components rs _q ^→ (ω, k) = [rsA _q (ω, k), rsB _q (ω, k)] (q = 1,..., Q) are obtained. . When δ = 1 is set, only the direction of arrival can be obtained as reflected sound information by not correcting the coefficient.

<< Second Embodiment >>
The process of step S5 in 2nd Embodiment is demonstrated. In the process of step S5 in the second embodiment, “§1.1 Initial value setting of direction information” is different from the first embodiment. Therefore, the duplicated description of the same items as those in the first embodiment will be omitted, and items different from those in the first embodiment will be described.

§1.1 Initial value setting of direction information First, initial values θ _{ini, q} ^→ (ω, k) (1 ≦ q ≦ Q) of Q pieces of direction information θ _q ^→ (ω, k) are set. As a method for determining Q initial values θ _{ini, q} ^→ (ω, k) (1 ≦ q ≦ Q), in this embodiment, initial values θ _{ini of} Q direction information θ _q ^→ (ω, k). _{, q} ^→ (ω, k) (1 ≦ q ≦ Q) will be described using the observation signal X ^→ (ω, k) and the template information S ^→ (ω). According to this method, Q templates corresponding to Q directions considered to be closest to each of Q estimated directions of arrival are determined, and direction information corresponding to the determined Q templates is determined. The initial value θ _{ini, q} ^→ (ω, k) of Q pieces of direction information θ _q ^→ (ω, k) may be set to 1 ≦ q ≦ Q. In this case, there is a relationship of Q <P between P and Q.

Therefore, in order to determine the template as described above from the template information, for convenience, the q-th reflected sound is expressed as A _q (ω, k, g (ω, q)) S _{g (ω, q)} ^→ (ω ). Here, g (ω, q) represents an index of a template that can represent the q-th reflected sound most accurately in the template information. The coefficient A _q (ω, k, g (ω, q)) constituting the reflected sound is determined by the template S _{g (ω, q)} ^→ (ω ) And the reflected sound. In this case, the residual signal E _{q + 1} ^→ (ω, k) obtained by removing q reflected sounds from the first to the qth from the observed signal is expressed as in Expression (20). However, 1 ≦ q ≦ Q, and E ₁ ^→ (ω, k) = X ^→ (ω, k).

The qth reflected sound A _q (ω, k, g (ω, q)) S _{g (ω, q)} ^→ (ω) is the residual signal E _{q + 1} ^→ (ω, k) based on equation (20). ) Power (E _{q + 1} ^→ (ω, k)) ^H E _{q + 1} ^→ (ω, k) is estimated according to a standard that minimizes. There are various estimation methods, but one of them will be described. The reflected sound consists of two elements, A _q (ω, k, g (ω, q)) and S _{g (ω, q)} ^→ (ω). Is required. <Process 1> and <Process 2> to be described later are performed for each q in ascending order of q.

<Process 1>
The symbol Λ is a set obtained by excluding the set of indexes determined by Equation (22) described later from the entire set {1,..., P,. That is, Λ = {1, ..., p, ..., P}-{g (ω, 1), ..., g (ω, q-1)}. However, when <Process 1> is performed for the first time, Λ = {1,..., P,.
The p-th (p∈Λ) template S _p ^→ (ω) is the power (E _{q + 1} ^→ (ω, k)) ^{H of the} residual signal E _{q + 1} ^→ (ω, k) based on the equation (20). The coefficient A _q (ω, k, p) when assuming that E _{q + 1} ^→ (ω, k) is an optimal template is obtained from the equation (21) based on the least square method. It is done. Note that at this stage, q on the left side of Equation (21) has no meaning.

<Process 2>
When the number (concentration) of elements of the set Λ is | Λ |, among the | Λ | coefficients A _q (ω, k, p) (p∈Λ) obtained based on the equation (21), The index q (1 ≦ q ≦ Q) of the coefficient A _q (ω, k, p) having the maximum absolute value is determined (see Expression (22)).

Accordingly, the initial value θ _{ini, q} ^→ (ω, k) (1 ≦ q ≦ Q) of the Q pieces of direction information θ _q ^→ (ω, k) is expressed by Q pieces of g (ω , q) (S ≦ _G ≦ ω ≦ q ≦ Q) (1 ≦ q ≦ Q) ^→ (ω) (1 ≦ q ≦ Q) direction information θ _{g (ω, q)} ^→ (ω) = [θ _{g (ω, q), pol} (ω), θ _{g (ω, q), azi} (ω)] (1 ≦ q ≦ Q). That is, θ _{ini, q} ^→ (ω, k) = [θ _{g (ω, q), pol} (ω), θ _{g (ω, q), azi} (ω)] (1 ≦ q ≦ Q). Note that the initial value θ _{ini, q} ^→ (ω, k) does not depend on the frame index k.

In the second embodiment, the initial value θ _{ini, q} ^→ (ω, k) (1 ≦ 1) of Q pieces of direction information θ _q ^→ (ω, k) is used instead of the beam search determination method used in the first embodiment. A generalized harmonic analytical determination method was used to determine q ≦ Q). According to the beam search determination method, there is a possibility that the initial value is biased due to the influence of the reflected sound having strong power, and if the initial value is biased, the estimation accuracy may be deteriorated. On the other hand, according to the generalized harmonic analysis determination method, the initial value calculation amount increases as compared with the beam search determination method, but it is highly possible to set an initial value close to the reflected sound (correct answer) to be estimated, In this case, not only can the estimation accuracy be improved, but the number of iterations can be reduced.

<< Third Embodiment >>
The process of step S5 in the third embodiment will be described. In the process of step S5 in the third embodiment, “§1.1 Initial value setting of direction information” is different from the first embodiment. Therefore, the duplicated description of the same items as those in the first embodiment will be omitted, and items different from those in the first embodiment will be described. The concept of the third embodiment is to prevent the bias of the initial values described above while being simpler than the second embodiment. In particular, the third embodiment is effective when there is a bias in the direction information corresponding to each template included in the template information used in the first embodiment. Even if the direction information corresponding to each template included in the template information used in the first embodiment is not biased, the direction information corresponding to the adjacent template is set as an initial value when the number of templates is extremely large. As described later, the direction information corresponding to the template used for setting the initial value is restricted so as not to be biased and sparse so as to prevent the bias of the initial value. Can do.

§1.1 Initial value setting of direction information First, a plurality of directions are determined so that there is no bias for either one of θ _pol (ω) and θ _azi (ω) constituting the direction information. In this example, β (β ≧ 2) directions {θ _{1, pol} (ω),..., Θ _{β, pol} (ω)} are determined so that there is no deviation with respect to the polar angle θ _pol (ω). Usually, the polar angle satisfies 0 ° ≦ θ _pol (ω) ≦ 180 °. For example, by setting the direction at equal intervals of 10 °, {θ _{1, pol} (ω),…, θ _{β, pol} (ω) } = {0,10,20, ..., 180} (β = 19). Corresponds to the direction information (position) having one of the elements of the set {θ _{1, pol} (ω),…, θ _{β, pol} (ω)} as the polar angle θ _pol (ω) among the templates included in the template information Let Ψ be the set of template indexes to be performed. At this time, it is preferable that the number (concentration) | Ψ | of the elements of the set Ψ satisfies Q ≦ | Ψ | <P. The set Ψ is a true subset of the set of direction information (position) corresponding to the template included in the template information.

Then, the coefficient A _q (ω, k, p) is obtained based on the equation (12). However, the index p of the template used on the right side of Expression (12) is pεΨ.

Next, out of | Ψ | coefficients A _q (ω, k, p) (1 ≦ p ≦ | Ψ |) obtained based on the equation (12) under the condition of p∈Ψ, The index q (1 ≦ q ≦ Q) of the Q coefficients A _q (ω, k, p) is determined in order from the largest value (see equation (23)). The symbol Γ is a set obtained by subtracting the set of indexes determined by the equation (23) from the set Ψ, and Γ = Ψ− {g (ω, 1),..., G (ω, q−1)}. .

Accordingly, the initial value θ _{ini, q} ^→ (ω, k) (1 ≦ q ≦ Q) of the Q pieces of direction information θ _q ^→ (ω, k) is expressed by Q pieces of g (ω , q) (S ≦ _G ≦ ω ≦ q ≦ Q) (1 ≦ q ≦ Q) ^→ (ω) (1 ≦ q ≦ Q) direction information θ _{g (ω, q)} ^→ (ω) = [θ _{g (ω, q), pol} (ω), θ _{g (ω, q), azi} (ω)] (1 ≦ q ≦ Q). That is, θ _{ini, q} ^→ (ω, k) = [θ _{g (ω, q), pol} (ω), θ _{g (ω, q), azi} (ω)] (1 ≦ q ≦ Q). Note that the initial value θ _{ini, q} ^→ (ω, k) does not depend on the frame index k.

In addition to these embodiments, an embodiment in which the initial value θ _{ini, q} ^→ (ω, k) (1 ≦ q ≦ Q) of Q pieces of direction information θ _q ^→ (ω, k) is set at random is allowed. The

<Modification>
In the first embodiment described above, the reflected sound information rs ^→ (ω, k) is estimated using the observation signal X ^→ (ω, k) for each frequency. However, when the reflected sound information is estimated for each frequency, the reflected sound information is estimated uniquely. Information on directions other than the direction of the virtual sound source to be performed (estimated arrival direction) may be included, and as a result, errors may occur in the reflected sound information. For example, it is desirable that only information related to the estimated arrival direction can be extracted as shown in FIG. 9A, but in reality, information related to directions other than the estimated arrival direction is mixed as shown in FIG. 9B. There can be.

  Therefore, in the modification, the estimation error of the reflected sound information is reduced by calculating the power collectively over all frequencies. That is, as shown in FIG. 10, the influence of directions other than the estimated arrival direction can be reduced as much as possible by integrating the power of the residual signal over all frequencies. In general, the power at each frequency varies in directions other than the estimated direction of arrival. Therefore, by integrating the power of the residual signal over all frequencies, the power in the other direction compared to the power in the estimated direction of arrival. The relative influence of power can be reduced. In FIG. 10, it should be noted that the scale of each graph is not the same because the power on the vertical axis indicates a relative value.

The processing in this modification is as follows. A set of frequency indexes ω included in the frequency band to be analyzed is Ω. For example, if an audio signal is handled, the set of indexes corresponding to the 1.0 to 3.0 kHz band may be Ω. Then, the index g (ω, q) of the template S _{g (ω, q)} ^→ (ω) is obtained by Expression (24) instead of Expression (13) or Expression (22). Similarly, it calculates | requires by Formula (25) instead of Formula (23). Further, the correction of the direction information θ _q ^→ (ω, k) = [θ _{q, pol} (ω, k), θ _{q, azi} (ω, k)] is performed by the equation (26) instead of the equation (16). Done by renewal.

<Application example>
The reflected sound information is very important voice information for human life. For example, a visually impaired person grasps the environment by reflecting a sound source signal generated by tapping on a wall or ceiling and observing with an ear. Further, even in everyday conversation, there is a difference in the ease of conversation between talking in a room where moderate reflection occurs and talking in an environment with relatively few reflected sounds. Hereinafter, service examples using reflected sound information estimated according to the present invention will be described.
The first is an example in which the present invention is incorporated in a conference system. Since the amplitude of the reflected sound changes according to the direction of the directional sound source, if the reflected sound information is known, it is possible to estimate in which direction the sound source is directed. If a sound source direction estimation device is incorporated in the conference system, it can be applied to presenting who spoke.
The second is a system that allows users to view video and audio at any position. Sound far away is difficult to pick up because the power of the sound source coming directly is small. If the reflected sound information is known, not only the direct sound but also the reflected sound can be picked up and collected, so that it is possible to enhance the sound in the distance. In the field of audio processing, it is possible to emphasize and collect sound sources by direction, but it is very difficult to emphasize and collect sounds by distance. If the reflected sound information is known, a physical feature amount corresponding to the distance can be obtained, so that sound can be collected for each distance. If far-field sounds can be picked up or picked up by direction and distance, a sound field corresponding to the position selected by the viewer can be generated in a pseudo manner.

  In a voice communication system, estimating reflected sound information leads to obtaining information on a sound field that could not be obtained only by direct sound. If the reflected sound information is known, it will lead to far-field sound collection and sound collection by distance, which could not be done with conventional speech enhancement technology, or information on the sound field that could not be estimated with conventional sound collection technology (for example, The direction of the sound source can be estimated. Such estimation of sound field information leads to the development of a speech processing apparatus that could not be realized by the conventional technology. The prior art related to the estimation of reflected sound information required observation of a special signal in order to obtain an impulse response, but the present invention has an advantage that reflected sound information can be obtained with a general observation signal such as an audio signal. have.

<Example of hardware configuration of reflected sound information estimation device>
The reflected sound information estimation apparatus according to the above-described embodiments may include an input unit to which a keyboard or the like can be connected, an output unit to which a liquid crystal display or the like can be connected, a CPU (Central Processing Unit) [cache memory, or the like. ] RAM (Random Access Memory) or ROM (Read Only Memory) and external storage device as a hard disk, and data exchange between these input unit, output unit, CPU, RAM, ROM, and external storage device It has a bus that can be connected. Further, if necessary, the reflected sound information estimation device may be provided with a device (drive) that can read and write a storage medium such as a CD-ROM. A physical entity having such hardware resources includes a general-purpose computer.

  The external storage device of the reflected sound information estimation device stores a program for estimating reflected sound information and data necessary for processing of the program [not limited to the external storage device, for example, a program is read-only. You may memorize | store in ROM which is a memory | storage device. ]. Data obtained by the processing of these programs is appropriately stored in a RAM or an external storage device. Hereinafter, a storage device that stores data, addresses of storage areas, and the like is simply referred to as a “storage unit”.

  The storage unit of the reflected sound information estimation device has a program for performing AD conversion on an analog signal, a program for performing frame division processing, and a program for converting a digital signal for each frame into an observation signal in the frequency domain A program for generating template information and a program for estimating reflected sound information using frequency domain observation signals and template information are stored.

  In the reflected sound information estimation apparatus, each program stored in the storage unit and data necessary for processing each program are read into the RAM as necessary, and are interpreted and processed by the CPU. As a result, the CPU realizes the predetermined functions (AD conversion unit, frame division unit, frequency domain conversion unit, template generation unit, reflection sound information estimation unit), thereby realizing the reflection sound information estimation.

<Supplementary note>
The present invention is not limited to the above-described embodiment, and can be appropriately changed without departing from the spirit of the present invention. In addition, the processing described in the above embodiment may be executed not only in time series according to the order of description but also in parallel or individually as required by the processing capability of the apparatus that executes the processing. .

  When the processing functions in the hardware entity (reflected sound information estimation apparatus) described in the above embodiment are realized by a computer, the processing contents of the functions that the hardware entity should have are described by a program. Then, by executing this program on a computer, the processing functions in the hardware entity are realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like, and as an optical disk, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only). Memory), CD-R (Recordable) / RW (ReWritable), etc., magneto-optical recording medium, MO (Magneto-Optical disc), etc., semiconductor memory, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. Can be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. When executing the process, the computer reads a program stored in its own recording medium and executes a process according to the read program. As another execution form of the program, the computer may directly read the program from a portable recording medium and execute processing according to the program, and the program is transferred from the server computer to the computer. Each time, the processing according to the received program may be executed sequentially. Also, the program is not transferred from the server computer to the computer, and the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by the execution instruction and result acquisition. It is good. Note that the program in this embodiment includes information that is used for processing by an electronic computer and that conforms to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer).

  In this embodiment, a hardware entity is configured by executing a predetermined program on a computer. However, at least a part of these processing contents may be realized by hardware.

Claims (15)

M in each of P directions (where P is a predetermined integer of 2 or more) viewed from a microphone array composed of M microphones (where M is an integer of 4 or more). A storage unit for storing a function simulating a transfer characteristic for each frequency (hereinafter referred to as a transfer characteristic function) as a template for each of the above microphones;
One of the emitted plurality of directions into the microphone array from the sound source (hereinafter, referred to as voice arrival direction) the audio signal coming from the M sound pickup signals respectively into the frequency domain obtained by sound pickup by M the microphone A converted signal (hereinafter referred to as an observation signal) is input, and Q is a predetermined integer of 2 or more, and one direct sound and Q-1 reflected sound (hereinafter referred to as the direct sound) are different from the direct sound. Each of the one direct sound and the Q-1 reflected sound, which is different from the direct sound, is referred to as Q sound signals) in the direction of voice arrival of the Q sound signals viewed from the microphone array. , for the M each said microphone, as represented by multiplying the complex amplitude to the transfer characteristic function for each frequency, the Q-number specified by the Q template selected from among the templates The voice arrival direction is set as the initial direction of each of the Q audio signals, the complex amplitude corresponding to each initial direction is determined as the initial amplitude, and the Q different initial directions and the Q initial amplitudes are used as starting points. residual signal obtained by subtracting the Q-number of audio signals corresponding to the number of the voice incoming direction from the observed signal (hereinafter, referred to as the residual signal after bulk removal) Q pieces of the audio arrives as the power of the smaller by performing the correction of the direction of correction and the Q of the complex amplitude, the Q of Q the sound arrival direction and the Q of the reflected sound Ru estimation Teisu collectively the complex amplitudes corresponding to the audio signal reflection sound information estimating apparatus including <br/> information estimation unit. The reflected sound information estimation apparatus according to claim 1 ,
The reflected sound information estimation unit
A voice arrival direction correction process for correcting the Q voice arrival directions and a complex amplitude correction process for correcting the Q complex amplitudes are alternately and repeatedly performed a predetermined number of times. The corresponding Q speech arrival directions and Q complex amplitudes are estimated collectively,
q is an integer between 1 and Q,
In the voice arrival direction correction process,
For each q , the q-th speech signal is added to the residual signal after batch removal (however, in the first speech arrival direction correction processing, the q-th speech signal is the initial direction of the q-th speech signal, For each of the M microphones, the transfer characteristic function for each frequency is multiplied by the initial amplitude of the q-th audio signal. In the second and subsequent voice arrival direction correction processes, the q-th audio signal Is the immediately preceding complex amplitude correction in the transfer characteristic function for each of the M microphones in the speech arrival direction of the q-th speech signal obtained in the immediately preceding speech arrival direction correction process. The q-th speech arrival direction is corrected so that the power of the signal obtained by adding the complex amplitude of the q-th speech signal obtained by the processing) is reduced ,
In the complex amplitude correction process,
For each q, the q-th speech signal is added to the residual signal after collective removal (however, in the first complex amplitude correction process, the q-th speech signal is obtained by the previous speech arrival direction correction process). Expressed by multiplying the transfer characteristic function for each of the M microphones in the voice arrival direction of the q-th audio signal by the initial amplitude of the q-th audio signal by the transfer characteristic function for each frequency. In the amplitude correction process, the q-th audio signal is the frequency-specific value for each of the M microphones in the audio arrival direction of the q-th audio signal obtained in the immediately preceding audio arrival direction correction process. the transfer characteristic function, as shortly before the complex amplitude correction process the q-th audio signal of the represented by multiplying the complex amplitude) was added to obtain a signal obtained by, the power is reduced q Eye reflection sound information estimating apparatus characterized by performing the correction of the complex amplitude. In the reflected sound information estimation device according to claim 2,
The reflected sound information estimation unit sets p as an integer between 1 and P,
(1) For each p, the power of the p-th residual signal obtained by subtracting the p-th template multiplied by the p-th complex amplitude (hereinafter referred to as virtual amplitude) from the observed signal is minimized. P-th virtual amplitude is determined, and among the P virtual amplitudes obtained as a result, Q speech arrivals corresponding to Q templates each multiplied by the top Q virtual amplitudes with respect to the magnitude are obtained. Let the direction be the Q initial directions,
(2) q as each integer of 1 or more Q or less, q-th of the initial direction to the q-th virtual amplitude so that the power becomes a minimum above Symbol removed together after remaining No. Sashin that Sadama corresponding And the Q virtual amplitudes are set as the Q initial amplitudes. In the reflected sound information estimation device according to claim 2,
The reflected sound information estimation unit
(1) A process for determining Q initial directions by performing a first process for obtaining a complex amplitude and a second process for obtaining a template index for each q in ascending order of q. Then, p represents each index included in the set obtained by excluding the template index determined in the second process from the set of all template indexes, and for each p, the pth complex is added to the pth template. The q + 1th residual signal obtained by subtracting the product of the amplitude (hereinafter referred to as virtual amplitude) from the qth minimum residual signal (where the first minimum residual signal is the observed signal). The p-th virtual amplitude is determined such that the power of the first virtual amplitude is minimized, and in the second process, the template obtained by multiplying the virtual amplitude obtained by the first process is multiplied Identify the index of the over bets, a voice arrival direction corresponding to the template by the q-th of the initial direction, to determine the Q-number of the initial direction,
(2) q as each integer of 1 or more Q or less, q-th of the initial direction to the q-th virtual amplitude so that the power becomes a minimum above Symbol removed together after remaining No. Sashin that Sadama corresponding And the Q virtual amplitudes are set as the Q initial amplitudes. In the reflected sound information estimation device according to claim 2,
The reflected sound information estimation unit
Determine the true subset Ψ of the voice arrival directions selected so as to be unbiased and sparse from the set of voice arrival directions corresponding to each of the templates, and set the concentration of the true subset Ψ to | Ψ | , Q ≦ | Ψ |, x represents each index of the template corresponding to the voice arrival direction included in the true subset Ψ,
(1) For each x, the power of the xth residual signal obtained by subtracting the xth template multiplied by the xth complex amplitude (hereinafter referred to as virtual amplitude) from the observed signal is minimized. The x-th virtual amplitude is determined for Q, and Q voice arrival directions corresponding to Q templates each of which is multiplied by the top Q virtual amplitudes with respect to the magnitude of the obtained virtual amplitude are defined as Q. And the above initial direction
(2) q as each integer of 1 or more Q or less, q-th of the initial direction to the q-th virtual amplitude so that the power becomes a minimum above Symbol removed together after remaining No. Sashin that Sadama corresponding And the Q virtual amplitudes are set as the Q initial amplitudes. In the reflected sound information estimation device according to any one of claims 1 to 5 ,
Each of the powers is a power obtained by adding over all the frequencies, and the reflected sound information estimation device. In the reflected sound information estimation device according to any one of claims 1 to 6 ,
Ω is the frequency, Ω is the set of frequencies ω, i is the imaginary unit, c is the speed of sound, the p th position [x _p , y _p , z _p ] and the m th (1 ≦ m ≦ M) microphone are arranged. S _pm (ω), the transfer characteristic between positions [u _m , v _m , w _m ], where

As described above, the reflected sound information estimation device further includes a template generation unit that generates each of the templates S _p (ω) = {S _p1 (ω),..., S _pM (ω)} (ω∈Ω ). . In the storage unit, there are P directions (where P is an integer greater than or equal to 2) as viewed from a microphone array including M (where M is an integer greater than or equal to 4) microphones. For each of the M microphones, a function that simulates a transfer characteristic for each frequency (hereinafter referred to as a transfer characteristic function) is stored as a template.
Reflection sound information estimation unit, one of the emitted plurality of directions into the microphone array from the sound source (hereinafter, referred to as voice arrival direction) of the M obtained audio signal coming from the sound pickup by M the microphone yield Each sound signal is converted into a frequency domain (hereinafter referred to as an observation signal), and Q is a predetermined integer of 2 or more. One direct sound is different from the above direct sound by Q−1. Each of the reflected sounds of Q (hereinafter referred to as Q sound signals by combining Q-1 reflected sounds that are different from the one direct sound and the direct sound) is Q sounds as viewed from the microphone array. voice arrival direction of the signal, for the M each said microphone, as represented by multiplying the complex amplitude to the transfer characteristic function for each frequency, the Q-number of template selected from among the templates The Q voice arrival directions specified are set as the initial directions of the Q voice signals, the complex amplitude corresponding to each initial direction is determined as the initial amplitude, and the Q initial directions and the Q initial amplitudes are determined. as a starting point, the different Q-number of the voice incoming corresponds to the direction the Q audio signal a residual signal obtained by subtracting from the observed signal (hereinafter, referred to as the residual signal after bulk removal) so that the power of the will decrease Q correction of the voice arrival direction and Q correction of the complex amplitude are performed at the same time, so that the Q voice arrival directions and the Q complex amplitudes corresponding to the Q voice signals are collectively obtained. reflection sound information estimation method having the estimated Teisu Ru reflection sound information estimation process Te. The reflected sound information estimation method according to claim 8 ,
In the reflected sound information estimation process,
A voice arrival direction correction process for correcting the Q voice arrival directions and a complex amplitude correction process for correcting the Q complex amplitudes are alternately and repeatedly performed a predetermined number of times, thereby each of the Q voice signals. Q speech arrival directions and Q complex amplitudes corresponding to are collectively estimated,
q is an integer between 1 and Q,
In the voice arrival direction correction process,
For each q , the q-th speech signal is added to the residual signal after batch removal (however, in the first speech arrival direction correction processing, the q-th speech signal is the initial direction of the q-th speech signal, For each of the M microphones, the transfer characteristic function for each frequency is multiplied by the initial amplitude of the q-th audio signal. In the second and subsequent voice arrival direction correction processes, the q-th audio signal Is the immediately preceding complex amplitude correction in the transfer characteristic function for each of the M microphones in the speech arrival direction of the q-th speech signal obtained in the immediately preceding speech arrival direction correction process. the q-th audio signal of the represented by multiplying the complex amplitude) was added resulting signals obtained in the process, the power of the q-th said sound arrival direction so as to reduce the correction performed
In the complex amplitude correction process,
For each q, the q-th speech signal is added to the residual signal after collective removal (however, in the first complex amplitude correction process, the q-th speech signal is obtained by the previous speech arrival direction correction process). Expressed by multiplying the transfer characteristic function for each of the M microphones in the voice arrival direction of the q-th audio signal by the initial amplitude of the q-th audio signal by the transfer characteristic function for each frequency. In the amplitude correction process, the q-th audio signal is the frequency-specific value for each of the M microphones in the audio arrival direction of the q-th audio signal obtained in the immediately preceding audio arrival direction correction process. the transfer characteristic function, as shortly before the complex amplitude correction process the q-th audio signal of the represented by multiplying the complex amplitude) was added to obtain a signal obtained by, the power is reduced q Reflection sound information estimating method comprising <br/> that eye correction of the complex amplitude is performed. In the reflected sound information estimation method according to claim 9,
In the reflected sound information estimation process, p is an integer between 1 and P,
(1) For each p, the power of the p-th residual signal obtained by subtracting the p-th template multiplied by the p-th complex amplitude (hereinafter referred to as virtual amplitude) from the observed signal is minimized. P-th virtual amplitude is determined, and among the P virtual amplitudes obtained as a result, Q speech arrivals corresponding to Q templates each multiplied by the top Q virtual amplitudes with respect to the magnitude are obtained. Let the direction be the Q initial directions,
(2) q as each integer of 1 or more Q or less, q-th of the initial direction to the q-th virtual amplitude so that the power becomes a minimum above Symbol removed together after remaining No. Sashin that Sadama corresponding And the Q virtual amplitudes are set as the Q initial amplitudes. In the reflected sound information estimation method according to claim 9,
In the reflected sound information estimation process,
(1) A process for determining Q initial directions by performing a first process for obtaining a complex amplitude and a second process for obtaining a template index for each q in ascending order of q. Then, p represents each index included in the set obtained by excluding the template index determined in the second process from the set of all template indexes, and for each p, the pth complex is added to the pth template. The q + 1th residual signal obtained by subtracting the product of the amplitude (hereinafter referred to as virtual amplitude) from the qth minimum residual signal (where the first minimum residual signal is the observed signal). The p-th virtual amplitude is determined such that the power of the first virtual amplitude is minimized, and in the second process, the template obtained by multiplying the virtual amplitude obtained by the first process is multiplied Identify the index of the over bets, a voice arrival direction corresponding to the template by the q-th of the initial direction, to determine the Q-number of the initial direction,
(2) q as each integer of 1 or more Q or less, q-th of the initial direction to the q-th virtual amplitude so that the power becomes a minimum above Symbol removed together after remaining No. Sashin that Sadama corresponding And the Q virtual amplitudes are set as the Q initial amplitudes. In the reflected sound information estimation method according to claim 9,
In the reflected sound information estimation process,
Determine the true subset Ψ of the voice arrival directions selected so as to be unbiased and sparse from the set of voice arrival directions corresponding to each of the templates, and set the concentration of the true subset Ψ to | Ψ | , Q ≦ | Ψ |, x represents each index of the template corresponding to the voice arrival direction included in the true subset Ψ,
(1) For each x, the power of the xth residual signal obtained by subtracting the xth template multiplied by the xth complex amplitude (hereinafter referred to as virtual amplitude) from the observed signal is minimized. The x-th virtual amplitude is determined for Q, and Q voice arrival directions corresponding to Q templates each of which is multiplied by the top Q virtual amplitudes with respect to the magnitude of the obtained virtual amplitude are defined as Q. And the above initial direction
(2) q as each integer of 1 or more Q or less, q-th of the initial direction to the q-th virtual amplitude so that the power becomes a minimum above Symbol removed together after remaining No. Sashin that Sadama corresponding And the Q virtual amplitudes are set as the Q initial amplitudes. In the reflected sound information estimation method according to any one of claims 8 to 12 ,
The reflected sound information estimation method, wherein each of the powers is a power obtained by adding over all the frequencies. In the reflected sound information estimation method according to any one of claims 8 to 13 ,
The template generation unit sets ω as the frequency, Ω as the set of frequencies ω, i as the imaginary unit, c as the speed of sound, p-th position [x _p , y _p , z _p ] and m-th (1 ≦ m ≦ M) S _pm (ω) is the transfer characteristic between the microphone location [u _m , v _m , w _m ], where

The method for estimating reflected sound information further includes a template generation process for generating each of the templates S _p (ω) = {S _p1 (ω),..., S _pM (ω)} (ω∈Ω ). . The computer program for executing the processing of the reflected sound information estimating method according to any of claims 8 to claim 14.