US12120490B2 - Filter coefficient optimization apparatus, filter coefficient optimization method, and program - Google Patents

Abstract

Provided is a filter coefficient optimization technology that makes it possible to design a stable beamformer having a good quality by considering the relationship of a filter coefficient between adjacent frequency bins. A filter coefficient optimization apparatus includes an optimization unit that calculates an optimum value of a filter coefficient w={w₁, . . . , w_F} (w_f is a filter coefficient of a frequency bin f) of a beamformer that emphasizes sound (target sound) from D sound source, a_f,d being an array manifold vector in the frequency bin f corresponding to a sound wave that comes from an angular direction θ_d in which a sound source d exists, the sound wave being a plane wave, the optimization unit calculating the optimum value based on an optimization problem of a cost function defined using a sum of a sum of a cost function L_{MV_f}(w_f) and a predetermined regularization term, under a predetermined constraint condition, the predetermined regularization term being defined using a difference in phase between adjacent frequency bins relevant to a response w_f ^Ha_f,d of the beamformer in the frequency bin f for the angular direction θ_d.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a U.S. National Stage Application filed under 35 U.S.C. § 371 claiming priority to International Patent Application No. PCT/JP2020/008233, filed on 28 Feb. 2020, the disclosure of which is hereby incorporated herein by reference in its entirety.

TECHNICAL FIELD

The present invention relates to a technology for optimizing a filter coefficient in target sound emphasis.

BACKGROUND ART

A beamforming using a microphone array is well known as a signal processing technique for emphasizing only sound (hereinafter referred to as target sound) that comes from a particular angular direction and suppressing sound (hereinafter referred to as non-target sound) that comes from other angular directions. This technique has been put to practical use in a telephone meeting system, a communication system in an automobile, a smart speaker, and the like.

In many conventional techniques relevant to the beamforming, an optimum filter is derived by solving an optimization problem of a cost function under some sort of constraint. For example, an MVDR (Minimum Variance Distortionless Response) beamformer described in Non Patent Literature 1 is obtained by using the power of an output signal as a cost function and minimizing this under a distortionless constraint condition for a target sound source angular direction.

Further, techniques of suppressing the non-target sound while imposing a constraint relevant to responses for a plurality of sound source directions in a situation where sound sources to be emphasized are in a plurality of angular directions have been already proposed as a beamformer design technique using a minimum variance method such as the technique described in Non Patent Literature 1. As one of them, there is an LCMV (Linearly Constrained Minimum Variance) beamformer (see Non Patent Literature 2). The LCMV beamformer emphasizes the target sound by imposing an equality constraint to responses of the beamformer for a plurality of angular directions, and suppresses the non-target sound by minimizing the variance of the output signal. A design technique for the LCMV beamformer will be described below in detail.

First, various definitions and notations are introduced. Hereinafter, signals are handled as values in time-frequency region after short-time Fourier transform.

A subscript of a time frame is expressed as t=1, . . . , T, and a subscript of a frequency bin is expressed as f=1, . . . , F. Further, complex conjugate transpositions of a vector v and a matrix M are expressed as a superscript ^H, as shown by v^H and M^H.

In the design of the LCMV beamformer, a linear filter (beamformer) that eliminate the non-target sound as unnecessary sound from an observation signal of a microphone array constituted by M microphone elements and emphasizes the target sound as the sound from a plurality of preset angular directions is configured. An observation signal for an M channel of the microphone array in a time frame t and a frequency bin f is shown as x_f,t∈C^M (f=1, . . . , F, t=1, . . . , T). A situation where D sound sources as signal sources that emit sound exist far off and a virtual plane wave comes to the microphone array is assumed. Further, it is assumed that all sound sources and all microphone elements are on identical planes. A signal that is emitted from a sound source d (d=1, . . . , D) and that comes to the microphone array in the time frame t and the frequency bin f is shown as s_d,f,t∈C (d=1, . . . , D, f=1, . . . , F, t=1, . . . , T). It is assumed that the sound of the sound source d comes from an angular direction θ_d. It is assumed that the angular direction θ_d is known.

When an array manifold vector (hereinafter referred to as an array manifold vector in the frequency bin f corresponding to a sound wave as a plane wave that comes from the angular direction θ_d) in the frequency bin f from the sound source d to M microphone elements of the microphone array is shown as a_f,d∈C^M (f=1, . . . , F, d=1, . . . , D), the observation signal x_f,t is expressed by the following expression.

$\begin{array}{cc}\left[\mathrm{Math}.1\right]& \\ x_{f,t}=\stackrel{D}{\sum _{d=1}}{s}_{d,f,t}{a}_{f,d}+n_{f,t}& \left(1\right)\end{array}$

Here, n_f,t (f=1, . . . , F, t=1, . . . , T) expresses a noise component including noises added in the course of the observation and other echoes and non-directional noises. The array manifold vector a_f,d is a quantity that is automatically determined for each frequency bin f from physical characteristics of the microphone array and the whole system.

Hereinafter, a linear filter in the frequency bin f is expressed as w_f∈C^M (f=1, . . . , F), and this is referred to as a filter coefficient of the beamformer. The filter coefficient determines the behavior of the beamformer.

An output signal y_f,t (f=1, . . . , F, t=1, . . . , T) of the beamformer is expressed by the following expression.
[Math. 2]
y _f,t =w _f ^H x _f,t  (2)

That is, the design of the beamformer is the design of a filter coefficient w_f (f=1, . . . , F) that meets Expression (2).

An inner product w_f ^Ha_f,d of the filter coefficient w_f and the array manifold vector a_f,d means a response characteristic of the beamformer in the frequency bin f for the angular direction θ_d. Accordingly, in a situation where it is desirable to certainly collect, at a constant gain, the sound that comes from a sound source in the angular direction θ_d (that is, from the sound source d), a method of imposing the following constraint condition (referred to as a distortionless constraint condition) on the filter coefficient w_f is often used.
[Math. 3]
w _f ^Hα_f,d=1  (3)

(f=1, . . . , F)

It is possible to achieve the emphasis of the sound that comes from the sound source d, by setting the filter coefficient w_f such that the distortionless constraint condition is met and gains for signals from unnecessary sound sources are reduced as much as possible.

In the case where it is desirable to concurrently emphasize the sound that comes from a plurality of sound sources, it is only necessary to concurrently impose a plurality of distortionless constraint conditions.

Since the beamformer is required to suppress the non-target sound, it is desired to set the filter coefficient w_f such that the non-target sound is minimized under the constraint of the target sound emphasis. For mathematically formulating this, a cost function expressing the variance of the non-target sound is defined. It is expected that it is possible to design a desired beamformer by setting the filter coefficient such that the cost function is minimized.

When a spatial correlation matrix R_f (f=1, . . . , F) of the non-target sound is defined as R_f:=E_t[x_f,tx_f,t ^H], a cost function L_{MV_f}(w_f) expressing the variance of the non-target sound can be defined for each of the frequency bins f=1, . . . , F. Specifically, the cost function L_{MV_f}(w_f) is shown as the following expression.
[Math. 4]
L _{MV f} (w _f)=w _f ^H R _f w _f  (4)

It is possible to design the beamformer by setting the filter coefficient w_f (f=1, . . . , F) such that the sum of the cost function L_{MV_f}(w_f) is minimized under the constraint condition in Expression (3). When this is expressed as a mathematical expression, an optimization problem in the following expression is obtained.

$\begin{array}{cc}\left[\mathrm{Math}.5\right]& \\ \underset{w_{}}{\mathrm{<figure-callout\; id="1"\; label="min\; w"\; filenames="US12120490-20241015-D00000.png,US12120490-20241015-D00001.png"\; state="\{\{state\}\}">min</figure-callout>}}\sum _f{L}_{{\mathrm{MV}}_f}\left(w_f\right)s.t.w_f^H{a}_{f,d}=1\left(f=1,\dots ,F,d=1,\dots ,D\right)& \left(5\right)\end{array}$

By solving the optimization problem in Expression (5), it is possible to obtain the optimum filter coefficient.

The optimization problem in Expression (5) can be divided into individual optimization problems for the respective frequency bins f=1, . . . , F. That is, for the frequency bin f, an optimization problem in the following expression may be solved instead of the optimization problem in Expression (5).

$\begin{array}{cc}\left[\mathrm{Math}.6\right]& \\ \underset{w_f}{\min }{L}_{{\mathrm{MV}}_f}\left(w_f\right)s.t.w_f^H{a}_{f,d}=1\left(d=1,\dots ,D\right)& \left(6\right)\end{array}$

By solving the optimization problem in Expression (5) or Expression (6) described above, it is possible to design the LCMV beamformer. This is the conventional design technique for the LCMV beamformer.

CITATION LIST Non-Patent Literature

• Non-Patent Literature 1: J. Capon, “High-resolution frequency-wavenumber spectrum analysis”, Proceedings of the IEEE, vol. 57, no. 8, pp. 1408-1418, August 1969.
• Non-Patent Literature 2: Futoshi Asano, “Acoustic Technology Series 16, Array signal processing for acoustics: localization, tracking and separation of sound sources, edited by The Acoustical Society of Japan”, Corona Publishing Co., Ltd., pp. 86-90, 2011.

SUMMARY OF THE INVENTION Technical Problem

In the conventional design technique for the LCMV beamformer, it is necessary to solve the optimization problem in Expression (5).

However, in the optimization problem in Expression (5), the relationship of the filter coefficient between adjacent frequency bins is not considered, and specifically, the reduction in the phase difference between adjacent frequency bins is not considered, so that it is not possible to design a stable beamformer having a good quality.

Hence, the present invention has an object to provide a filter coefficient optimization technology that makes it possible to design a stable beamformer having a good quality by considering the relationship of the filter coefficient between adjacent frequency bins.

Means for Solving the Problem

An aspect of the present invention is a filter coefficient optimization apparatus including an optimization unit that calculates an optimum value w* of a filter coefficient w={w₁, . . . , w_F} (w_f (f=1, . . . , F, F is an integer equal to or more than 1) is a filter coefficient of a frequency bin f) of a beamformer that emphasizes sound (hereinafter referred to as target sound) from D sound sources (hereinafter referred to as a sound source 1, . . . , a sound source D), D being an integer equal to or more than 1, R_f (f=1, . . . , F) being a spatial correlation matrix for sound other than the target sound relevant to the frequency bin f, L_{MV_f}(w_f)=w_f ^HR_fw_f (f=1, . . . , F) being a cost function relevant to a filter coefficient w_f, θ_d (d=1, . . . , D) being an angular direction in which a sound source d exists, a_f,d (f=1, . . . , F, d=1, . . . , D) being an array manifold vector in the frequency bin f corresponding to a sound wave that comes from the angular direction θ_d, the sound wave being a plane wave, L(w) being a cost function relevant to the filter coefficient w and being defined using a sum of a sum Σ_f=1 ^FL_{MV_f}(w_f) of the cost function L_{MV_f}(w_f) and a predetermined regularization term, the optimization unit calculating the optimum value w* based on an optimization problem min_wL(w) relevant to the filter coefficient w, under a predetermined constraint condition, the predetermined regularization term being defined using a difference in phase between adjacent frequency bins relevant to a response w_f ^Ha_f,d (f=1, . . . , F, d=1, . . . , D) of the beamformer in the frequency bin f for the angular direction θ_d.

An aspect of the present invention is a filter coefficient optimization apparatus including an optimization unit that calculates an optimum value w* of a filter coefficient w={w₁, . . . , w_F} (w_f (f=1, . . . , F, F is an integer equal to or more than 1) is a filter coefficient of a frequency bin f) of a beamformer that emphasizes sound (hereinafter referred to as target sound) from D sound sources (hereinafter referred to as a sound source 1, . . . , a sound source D), D being an integer equal to or more than 1, θ_d (d=1, . . . , D) being an angular direction in which a sound source d exists, a_f,d (f=1, . . . , F, d=1, . . . , D) being an array manifold vector in the frequency bin f corresponding to a sound wave that comes from the angular direction θ_d, the sound wave being a plane wave, the optimization unit calculating the optimum value w* by performing derivation so as to reduce a difference in phase between adjacent frequency bins relevant to a response w_f ^Ha_f,d (f=1, . . . , F, d=1, . . . , D) of the beamformer in the frequency bin f for the angular direction θ_d.

Effects of the Invention

According to the present invention, it is possible to design a stable beamformer having a good quality, by optimizing the filter coefficient in consideration of the relationship of the filter coefficient between adjacent frequency bins.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram showing a filter coefficient optimization algorithm.

FIG. 2 is a diagram showing a latent variable optimization algorithm.

FIG. 3 is a diagram showing a filter coefficient optimization algorithm.

FIG. 4 is a diagram showing a filter coefficient optimization algorithm.

FIG. 5 is a block diagram showing the configuration of a filter coefficient optimization apparatus 100.

FIG. 6 is a flowchart showing the behavior of the filter coefficient optimization apparatus 100.

FIG. 7 is a block diagram showing the configuration of an optimization unit 120.

FIG. 8 is a flowchart showing the behavior of the optimization unit 120.

FIG. 9 is a diagram showing an example of the functional configuration of a computer that realizes apparatuses in embodiments of the present invention.

DESCRIPTION OF EMBODIMENTS

Embodiments of the present invention will be described below in detail. Component units having identical functions are denoted by identical numerals, and repetitive descriptions are omitted.

Before the description of the embodiments, the notation method in the specification will be described.

“_” (underscore) indicates an inferior subscript. For example, “x^y_z” shows that “y_z” is a superscript for “x”, and “x_{y_z}” shows that “y_z” is an inferior subscript for “x”.

Further, for a certain character “x”, superscripts “{circumflex over ( )}” and “˜” for “{circumflex over ( )}x” and “˜x” should be originally put just above “x”, but “{circumflex over ( )}x” and “˜x” are shown because of the constraint about the notation in the specification.

<Technical Background>

A cost term (hereinafter referred to as a regularization term) in which the relationship of the filter coefficient between adjacent frequency bins is considered can be used for designing a stable beamformer having a good quality. In this technique, a new cost function is introduced by adding the regularization term to the cost function Σ_fL_{MV_f}(w_f) described in Background Art, and the filter coefficient is determined by solving an optimization problem of the new cost function. When a first order difference and a second order difference in a frequency direction of a phase component related to the filter coefficient are used as the relationship of the filter coefficient between adjacent frequency bins, it is expected that it is possible to design a filter having a stable delay characteristic by using a regularization term corresponding to the first order difference and the second order difference, because the first order difference and the second order difference correspond to a phase delay and a group delay respectively.

The reason why the simple difference in the filter coefficient is not used as the relationship of the filter coefficient between adjacent frequency bins is that the calculation amount can become enormous. For avoiding this problem, in the invention in the present patent application, attention is focused on the difference in phase component related to the filter coefficient, in consideration of the correspondence to the above phase delay and group delay.

<<Regularization Term by Frequency-Directional Smoothing>>

Here, a regularization term using the difference in the phase component related to the filter coefficient will be descried as a regularization term by frequency-directional smoothing. The regularization term makes it possible to directly control the group delay and phase delay of the filter constituting the beamformer.

The response of the beamformer in the frequency bin f for the angular direction θ_d is expressed as a complex number w_f ^Ha_f,d. An absolute value |w_f ^Ha_f,d| of the response w_f ^Ha_f,d of the beamformer is referred to as an amplitude, and a deflection angle ∠(w_f ^Ha_f,d) is referred to as a phase. Two forms will be shown below as examples of the regularization term by the frequency-directional smoothing.

(Continuous Form)

As an example of the regularization term in the form, there is a regularization term that is defined by the first order difference in phase between adjacent frequency bins. This regularization term is given by the following expression.

$\begin{array}{cc}\left[\mathrm{Math}.7\right]& \\ \eta \stackrel{F-1}{\sum _{f=1}}\stackrel{D}{\sum _{d=1}}{❘\angle \left(w_f^H{a}_{f,d}\right)-\angle \left(w_{f+1}^H{a}_{f+1,d}\right)❘}_{2\pi }& \left(7\right)\end{array}$

Further, as another example, there is a regularization term that is defined by the second order difference in phase between adjacent frequency bins. This regularization term is given by the following expression.

$\begin{array}{cc}\left[\mathrm{Math}.8\right]& \\ \eta \stackrel{F-2}{\sum _{f=1}}\stackrel{D}{\sum _{d=1}}{❘\angle \left(w_f^H{a}_{f,d}\right)-2\angle \left(w_{f+1}^H{a}_{f+1,d}\right)+\angle \left(w_{f+2}^H{a}_{f+2,d}\right)❘}_{2\pi }& \left(8\right)\end{array}$

In Expression (7) and Expression (8), η (η is a predetermined positive number) represents a weight parameter. Further, |●|_2π in Expression (7) and Expression (8) is a norm that is defined by the following expression.

$\begin{array}{cc}\left[\mathrm{Math}.9\right]& \\ {❘x❘}_{2\pi }=\underset{n=\cdots ,-1,0,1,\cdots }{\min }❘x-2n\pi ❘& \left(9\right)\end{array}$

That is, |x|_2π is a special norm in which the periodicity of a variable x is considered.

(Discrete Form)

For defining the regularization term in the form, a variable c_f,d (f=1, . . . , F, d=1, . . . , D) that depends on the phase of the response w_f ^Ha_f,d of the beamformer and that has discrete values is introduced for a filter coefficient w_f and an array manifold vector a_f,d. Specifically, a complex plane is divided into C sectors that are around the origin and that have an equal central angle, consecutive numbers 1, . . . , C are assigned in a counterclockwise manner, and c_f,d is the number of a sector where the complex number w_f ^Ha_f,d is positioned. Accordingly, the discrete variable c_f,d has one value of 1, . . . , C. Further, the following expression is satisfied among the filter coefficient w_f, the array manifold vector a_f,d and the discrete variable c_f,d.

$\begin{array}{cc}\left[\mathrm{Math}.10\right]& \\ \angle \left(w_f^H{a}_{f,d}\right)\in \left[2\pi \left(c_{f,d}-1\right)/C,2\pi c_{f,d}/C\right]& \left(10\right)\end{array}$

The regularization term in the following expression is defined using the discrete variable c_f,d that meets Expression (10).

$\begin{array}{cc}\left[\mathrm{Math}.11\right]& \\ \eta \stackrel{F-1}{\sum _{f=1}}\stackrel{D}{\sum _{d=1}}{❘c_{f,d}-c_{f+1,d}❘}_C& \left(11\right)\end{array}$

In Expression (11), η (η is a predetermined positive number) represents a weight parameter. Further, |●|_c in Expression (11) is a norm that is defined by the following expression.

$\begin{array}{cc}\left[\mathrm{Math}12\right]& \\ {❘x❘}_C=\underset{n=\dots ,-1,0,1,\dots }{\min }❘x-\mathrm{nC}❘& \left(12\right)\end{array}$

That is, |x|_c is a special norm in which the periodicity of the variable x is considered.

In the case of c_f=(c_f,1, . . . , c_f,D), the regularization term (hereinafter referred to as {circumflex over ( )}L_η(c_1,1, . . . , c_F,D)) in Expression (11) is expressed as follows.

$\begin{array}{cc}\left[\mathrm{Math}.13\right]& \\ {\hat{L}}_{\eta }\left({\mathrm{<figure-callout\; id="1"\; label="c"\; filenames="US12120490-20241015-D00000.png,US12120490-20241015-D00001.png"\; state="\{\{state\}\}">\; <figure-callout\; id="1"\; label="c"\; filenames="US12120490-20241015-D00000.png,US12120490-20241015-D00001.png"\; state="\{\{state\}\}">c</figure-callout>\; </figure-callout>}}_{1,1},\dots ,c_{F,D}\right)=\stackrel{F-1}{\sum _{f=1}}{\hat{L}}_{\eta f}\left(c_f,c_{f+1}\right)& \left(13\right)\end{array}$ $\begin{array}{cc}{\hat{L}}_{\eta f}\left(c_f,c_{f+1}\right)=\eta \stackrel{D}{\sum _{d=1}}{❘c_{f,d}-c_{f+1,d}❘}_C& \left(14\right)\end{array}$

An example of the introduction of the regularization term {circumflex over ( )}L_η(c_1,1, . . . , c_F,D) in the design of the LCMV beamformer will be described below. Assuming that a particularly important target sound exists in the first angular direction (that is, an angular direction θ₁) of D angular directions θ₁, . . . , θ_D, a distortionless constraint condition w_f ^Ha_f,1=1 (f=1, . . . , F) is imposed for the angular direction θ₁. In this case, an optimization problem to be solved is shown as follows.

$\begin{array}{cc}\left[\mathrm{Math}.14\right]& \\ \underset{{\left\{c_f,w_f\right\}}_{f=1}^F}{\min }\left(\stackrel{F}{\sum _{f=1}}{L}_{{\mathrm{MV}}_f}\left(w_f\right)+\stackrel{F-1}{\sum _{f=1}}{\hat{L}}_{\eta f}\left(c_f,c_{f+1}\right)\right)s.t.\angle \left(w_f^H{a}_{f,d}\right)\in \left[2\pi \left(c_{f,d}-1\right)/C,2\pi c_{f,d}/C\right]\left(f=1,\dots ,F,d=1,\dots ,D\right),w_f^H{a}_{f,1}=1\left(f=1,\dots ,F\right)& \left(15\right)\end{array}$

This optimization problem can be solved by evaluating optimum values of the filter coefficient w_f that minimize a cost function Σ_f=1 ^FL_{MV_f}(w_f)+Σ_f=1 ^F-1{circumflex over ( )}L_ηf(c_f, c_f+1) for all values that the discrete variable c_f (f=1, . . . , F) can have, and thereafter, among them, adopting the optimum value that minimizes the value of the cost function Σ_f=1 ^FL_{MV_f}(w_f)+Σ_f=1 ^F-1{circumflex over ( )}L_ηf(c_f, c_f+1), but in fact, there is a more efficient algorithm. The algorithm is shown in FIG. 1 .

When the value of the discrete variable c_f is set, the optimum value of the filter coefficient w_f is determined depending on only the value of the discrete variable c_f, regardless of the values of the other frequency bins. Therefore, by previously the filter coefficient w^f for all values that C^D discrete variables c_f can have for each frequency bin f, the optimization problem results in a shortest path problem relevant to the discrete variable c_f. Accordingly, the optimization problem can be solved at high speed by applying a Dijkstra method. This is used in the algorithm in FIG. 1 .

In the optimization problem in Expression (15), the distortionless constraint condition for one angular direction is used, but a distortionless constraint condition for a plurality of angular directions may be used. However, when the distortionless constraint condition for a plurality of angular directions is used, the constraint sometimes becomes excessively strict, so that the solution is not evaluated. The relaxation of the distortionless constraint condition is possible, but in this case, a non-convex optimization problem is sometimes obtained. Generally, it is difficult to solve the non-convex optimization problem. Hence, a technique for optimizing the filter coefficient by solving a convex optimization problem equivalent to the non-convex optimization problem instead of solving the non-convex optimization problem will be described below.

First, a method for transforming the non-convex optimization problem into the convex optimization problem equivalent to the non-convex optimization problem and a method for solving the convex optimization problem obtained by the transformation will be described. Next, two examples will be described as examples of the use of the method for the non-convex optimization problem obtained by the relaxation of the constraint condition.

<<Transformation into Convex Optimization Problem Equivalent to Non-Convex Optimization Problem and Solution Method>>

Here, a method for transforming the non-convex optimization problem into the convex optimization problem equivalent to the non-convex optimization problem and a method for solving the convex optimization problem obtained by the transformation will be described. An optimization problem relevant to a latent variable ˜w that is defined by the following expression will be discussed below.

$\begin{array}{cc}\left[\mathrm{Math}.15\right]& \\ \underset{\tilde{w}}{\min }\left(L_{\mathrm{convex}}\left(\tilde{w}\right)+\stackrel{D}{\sum _{d=1}}{L}_d\left(\tilde{w}\right)\right)& \left(16\right)\end{array}$

Here, L_convex is a strongly convex function relevant to the latent variable ˜w, and L_d (d=1, . . . , D, D is an integer equal to or more than 1) is a function relevant to the latent variable ˜w. That is, L_d (d=1, . . . , D) does not always need to be a convex function.

Generally, the optimization problem in Expression (16) is an optimization problem in which the cost function is a non-convex function, that is, a non-convex optimization problem. The non-convex optimization problem is a difficult problem as described above, and therefore, is intended to result in a convex optimization problem to be solved more easily, by introducing a certain kind of approximation. Hence, the function L_d(˜w) (d=1, . . . , D) is intended to be approximated by a piecewise convex function constituted by a plurality of convex functions.

The definition of the piecewise convex function will be described below. For the function L_d(˜w) (d=1, . . . , D) to be approximated, the domain is divided into regions S_d,1, . . . , S_d,C that are C closed convex sets. Then, a function {circumflex over ( )}_d,c (c=1, . . . , C) that is defined for each of the regions S_d,1, . . . , S_d,C is introduced. The newly introduced function {circumflex over ( )}_d,c is a convex function on the region S_d,c, and is a function for approximating the function L_d on the region S_d,c. In the case where the function L_d is a convex function on the region S_d,c, {circumflex over ( )}_d,c=L_d may be adopted on the region S_d,c. Thereby, the function L_d(˜w) can be approximately expressed by the piecewise convex function {circumflex over ( )}_d,c (c=1, . . . , C). Generally, as the value (that is, the number into which the domain of the function L_d is divided) of C is larger, the approximation can be performed by a more accurate piecewise convex function.

However, when the approximation is used, a discrete variable representing a region to which the optimum value as the solution of the optimization problem belongs is newly added as an optimized object, in addition to the latent variable that is an optimized object in the optimization problem in Expression (16), so that the number of variables to be optimized increases. However, when the discrete variable is fixed, for the latent variable, the optimization problem results in the convex optimization (instead of the non-convex optimization), and therefore can be solved relatively easily. This will be specifically described below. The optimization problem that is formulated using the approximation is expressed by the following expression, with c_d (d=1, . . . , D) as a discrete variable that has a value of 1, . . . , C.

$\begin{array}{cc}\left[\mathrm{Math}.16\right]& \\ \underset{\tilde{w}}{\min }\left(L_{\mathrm{convex}}\left(\tilde{w}\right)+\stackrel{D}{\sum _{d=1}}\underset{c_d}{\min }{\Lambda }_{d,c_d}\left(\tilde{w}\right)\right)& \left(17\right)\end{array}$

Expression (17) is equivalent to the following expression.

$\begin{array}{cc}\left[\mathrm{Math}.17\right]& \\ \underset{c_{}}{\mathrm{<figure-callout\; id="1"\; label="min\; c"\; filenames="US12120490-20241015-D00000.png,US12120490-20241015-D00001.png"\; state="\{\{state\}\}">min</figure-callout>}}\left(\underset{\tilde{w}}{\min }\left(L_{\mathrm{convex}}\left(\tilde{w}\right)+\stackrel{D}{\sum _{d=1}}{\Lambda }_{d,c_d}\left(\tilde{w}\right)\right)\right)& \left(18\right)\end{array}$

In Expression (18), min_˜w(L_convex(˜w)+Σ_d=1 ^D{circumflex over ( )}_{d,c_d}(˜w)) is a convex optimization problem relevant to the latent variable ˜w, and can be solved relatively easily. The procedure will be described below. First, the convex optimization problem min_˜w(L_convex (˜w)+Σ_d=1 ^D{circumflex over ( )}_{d,c_d}(˜w)) is solved for all values that the discrete variable (c₁, . . . , c_D) can have. Thereby, the solution of the convex optimization problem min_˜w(L_convex(˜w)+Σ_d=1 ^D{circumflex over ( )}_{d,c_d}(˜w)) is evaluated for all values that the C^D discrete variables (c₁, . . . , c_D) can have. Then, among the obtained solutions of the convex optimization problem, a solution that minimizes the value the cost function L_convex(˜w)+Σ_d=1 ^D{circumflex over ( )}_{d,c_d}(˜w) is adopted as the optimum value. Thereby, the optimization problem in Expression (18) can be solved. The procedure of the solution method is illustrated in FIG. 2 .

The non-convex optimization problem in Expression (16) can be transformed into the convex optimization problem in Expression (18) that is equivalent to the non-convex optimization problem in Expression (16), and the convex optimization problem in Expression (18) can be solved by the latent variable optimization algorithm in FIG. 2 .

Application Example

Here, an example in which the above-described versatile scheme of evaluating the optimum value after transforming the non-convex optimization problem into the convex optimization problem is applied to the non-convex optimization problem obtained by relaxing the constraint condition in Expression (3) will be described.

As described above, in the related art in Non Patent Literature 1, Expression (3) that is an equality constraint is imposed for many objects, and therefore, there is a fear that an appropriate filter coefficient cannot be obtained. Hence, it is intended to use a softer constraint condition that is suitable for a real situation. Specifically, it is intended to use a constraint condition (that is, a constraint condition in which there is no constraint relevant to the phase) in which a constraint is imposed for only the amplitude of the response of the beamformer, instead of the constraint condition in Expression (3). For example, the following expression can be used.
[Math. 18]
|w _f ^Hα_f,d|=1  (19)

Further, as another example, the following expression can be used.
[Math. 19]
|w _f ^Hα_f,d|≥1  (20)

The constraint condition in Expression (19) and the constraint condition in Expression (20) express the constraint that the amplitude of the response of the beamformer is a constant value (specifically, 1) and the constraint that the amplitude of the response of the beamformer only needs to be equal to or more than a constant value (specifically, 1), respectively. Each of the constraint condition in Expression (19) and the constraint condition in Expression (20) is mathematically classified into a non-convex constraint.

An optimization problem in which the constraint condition is Expression (20) will be discussed below. The constraint condition in Expression (20) shows that the absolute value of the complex number w_f ^Ha_f,d is equal to or more than 1. This means that the complex number w_f ^Ha_f,d needs to be geometrically positioned on a unit circle or outside the unit circle in the complex plane. Hence, first, the complex plane is equally divided into C sectors that are around the origin. The C sectors correspond to the C regions described above. Then, on the border or inside of each sector, Expression (20) that is the original constraint is approximated by C convex functions.

This will be specifically described below. The discrete variable c_f,d is adopted as a variable that has a value of 1, . . . , C, for the frequency bin f (f=1, . . . , F) and the sound source d (d=1, . . . , D). Further, γ_f,d=w_f ^Ha_f,d is satisfied. A convex function {circumflex over ( )}_{(f,d),c_f,d}(γ_f,d) (c_f,d=1, . . . , C) that is defined for the frequency bin f (f=1, . . . , F) and the sound source d (d=1, . . . , D) is defined such that the values of the complex number γ_f,d are restricted inside the sectors around the origin at a central angle 2π/C on the complex plane and in a range in which |γ_f,d|≥1 is met. Then, Expression (20) is approximated by a piecewise convex function using the C convex functions {circumflex over ( )}_{(f,d),c_f,d}(γ_f,d) (c_f,d=1, . . . , C).

For example, the function {circumflex over ( )}_{(f,d),c_f,d} may be a function expressed by the following expression.

$\left[\mathrm{Math}.20\right]$ $\begin{array}{cc}{\Lambda }_{\left(f,d\right)c_{f,d}}\left({\gamma }_{f,d}\right):=\{\begin{array}{c}0(R\left({\gamma }_{f,d}{e}^{-2\pi j\left(c_{f,d}+1\right)/2C}\ge 1,\frac{2\pi c_{f,d}}{C}\le {\mathrm{\angle \gamma }}_{f,d}\le \frac{2\pi \left(c_{f,d}+1\right)}{C}\right)\\ \infty \left(\mathrm{otherwise}\right)\end{array}& \left(21\right)\end{array}$

Here, R(z) represents the real part of a complex number z.

When the value of C is large, the approximation can be performed more accurately, but in the case of solving the optimization problem using the algorithm in FIG. 2 , it is necessary to examine all combinations of the discrete variables, so that the calculation amount increases.

Thus, the filter coefficient optimization problem in which the constraint condition is Expression (20) results in a convex optimization problem in the following expression.

$\begin{array}{cc}\left[\mathrm{Math}.21\right]& \\ \underset{{\left\{c_f,w_f\right\}}_{f=1}^F}{\min }\left(\stackrel{F}{\sum _{f=1}}{L}_{{\mathrm{MV}}_f}\left(w_f\right)+\stackrel{F}{\sum _{f=1}}\stackrel{D}{\sum _{d=1}}{\Lambda }_{\left(f,d\right),c_{f,d}}\left(w_f^H{a}_{f,d}\right)\right)& \left(22\right)\end{array}$

Here, c_f=(c_f,1, . . . , c_f,D) is satisfied.

This optimization problem can be solved by applying the latent variable optimization algorithm in FIG. 2 . An algorithm for solving the optimization problem is shown in FIG. 3 . That is, FIG. 3 shows a filter coefficient optimization algorithm that is obtained based on the latent variable optimization algorithm in FIG. 2 .

Application Example 2

The optimization problem of the filter coefficient w that is defined using the cost function Σ_f=1 ^FL_{MV_f}(w_f)+Σ_f=1 ^F-1{circumflex over ( )}L_ηf(c_f, c_f+1) under the constraint condition |w_f ^Ha_f,d|≥1 (f=1, . . . , F, d=1, . . . , D) will be discussed. This problem is a non-convex optimization problem that is obtained by using the constraint condition |w_f ^Ha_f,d|≥1 (f=1, . . . , F, d=1, . . . , D) instead of the constraint condition w_f ^Ha_f,d=1 (f=1, . . . , F, d=1, . . . , D).

Note that the discrete variable c_f,d defined in <<Regularization Term by Frequency-Directional Smoothing>> and the discrete variable c_f,d defined in <<Application Example>> are the same as each other. Thereby, the above non-convex optimization problem results in the following convex optimization problem.

$\left[\mathrm{Math}.22\right]$ $\begin{array}{cc}\underset{{\left\{c_f,w_f\right\}}_{f=1}^F}{\min }\left(\text{⁠}\stackrel{F}{\sum _{f=1}}{L}_{{\mathrm{MV}}_f}\left(w_f\right)+\stackrel{F}{\sum _{f=1}}\stackrel{D}{\sum _{d=1}}{\Lambda }_{\left(f,d\right),c_{f,d}}\left(w_f^H{a}_{f,d}\right)+\stackrel{F-1}{\sum _{f=1}}{\hat{L}}_{\eta f}\left(c_f,c_{f+1}\right)\right)& \left(23\right)\end{array}$

Similarly to the optimization problem in Expression (15), the optimization problem in Expression (23) can be solved at high speed by applying the Dijkstra method. The algorithm is shown in FIG. 4 .

First Embodiment

From a signal (observation signal) resulting from observing sound (hereinafter referred to as target sound) from D (D is an integer equal to or more than 1) sound sources (hereinafter referred to as a sound source 1, . . . , a sound source D), a filter coefficient optimization apparatus 100 calculates the optimum value w* of the filter coefficient w={w₁, . . . , w_F} (w_f (f=1, . . . , F, F is an integer equal to or more than 1) is the filter coefficient of the frequency bin f) of the beamformer that emphasizes the target sound, using a microphone array constituted by M (M is an integer equal to or more than 1) microphone elements. The observation signal is an input data that is used for the optimization of the filter coefficient, and therefore, the observation signal is referred to as optimization data, hereinafter.

The filter coefficient optimization apparatus 100 will be described below with reference to FIG. 5 and FIG. 6 . FIG. 5 is a block diagram showing the configuration of the filter coefficient optimization apparatus 100. FIG. 6 is a flowchart showing the behavior of the filter coefficient optimization apparatus 100. As shown in FIG. 5 , the filter coefficient optimization apparatus 100 includes a setup data calculation unit 110, an optimization unit 120, and a recording unit 190. The recording unit 190 is a component unit that appropriately records the information necessary for the processing in the filter coefficient optimization apparatus 100. For example, the recording unit 190 records the filter coefficient that is an optimized object.

The behavior of the filter coefficient optimization apparatus 100 will be described with FIG. 6 .

In S110, the setup data calculation unit 110 calculates setup data that is used at the time of the optimization of the filter coefficient w, using the optimization data. In the case of using the cost function for optimizing the filter coefficient w, examples of the setup data include a spatial correlation matrix R_f (f=1, . . . , F) for sound other than the target sound relevant to the frequency bin f and the array manifold vector a_f,d (f=1, . . . , F, d=1, . . . , D) in the frequency bin f corresponding to a sound wave as a plane wave that comes from the angular direction θ_d (d=1, . . . , D) in which the sound source d exists obtained based on the observation signal.

In S120, the optimization unit 120 calculates the optimum value w* of the filter coefficient w, using the setup data generated in S110. For example, the optimization unit 120 can calculate the optimization value w* based on the optimization problem min_wL(w) relevant to the filter coefficient w under a predetermined constraint condition. Here, L_{MV_f}(w_f)=w_f ^HR_fw_f (f=1, . . . , F) is a cost function relevant to the filter coefficient w_f, and the function L(w) is a cost function relevant to the filter coefficient w and is defined using the sum of the sum Σ_f=I ^FL_{MV_f}(w_f) of the function L_{MV_f}(w_f) and a predetermined regularization term. Further, the predetermined regularization term is a regularization term that is defined using the difference in phase between adjacent frequency bins relevant to the response w_f ^Ha_f,d (f=1, . . . , F, d=1, . . . , D) of the beamformer in the frequency bin f for the angular direction θ_d.

Some examples of the regularization term will be shown. Here, η is a predetermined positive value, and ∠(w_f ^Ha_f,d) (f=1, . . . , F, d=1, . . . , D) expresses the phase of the response w_f ^Ha_f,d of the beamformer in the frequency bin f for the angular direction θ_d.

The first example is ηΣ_f=1 ^F-1Σ_d=1 ^D|∠(w_f ^Ha_f,d)−∠(w_f+1 ^Ha_f+1,d)|_2π. The second example is ηΣ_f=1 ^F-2Σ_d=1 ^D|∠(w_f ^Ha_f,d)−2∠(w_f+1 ^Ha_f+1,d)+∠(w_f+2 ^Ha_f+2,d)|_2π.

The third example is ηΣ_f=1 ^F-1Σ_d=1 ^D|c_f,d−c_f+1,d|_c. Here, C is an integer equal to or more than 1, and c_f,d (f=1, . . . , F, d=1, . . . , D) is a discrete variable that has one value of 1, . . . , C that satisfies ∠(w_f ^Ha_f,d)∈[2π(c_f,d−1)/C, 2πc_f,d/C] for the phase ∠(w_f ^Ha_f,d).

A case where the third example is used as the regularization term will be described below. In this case, an example of the constraint condition is expressed by the following expression.
[Math. 23]
w _f ^Hα_f,1=1  (*)

(f=1, . . . , F)

Further, another example of the constraint condition is expressed by the following expression.
[Math. 24]
|w _f ^Hα_f,d|≥1  (**)

(f=1, . . . , F, d=1, . . . , D)

Case 1 and Case 2 will be described below. Case 1 is a case where the optimization unit 120 solves the optimization problem of the cost function that is defined using the sum of the sum Σ_f=1 ^FL_{MV_f}(w_f) of the cost function L_{MV_f}(w_f) and the regularization term ηΣ_f=1 ^F-1Σ_d=1 ^D|c_f,d−c_f+1,d|_c under a constraint condition (*). Case 2 is a case where the optimization unit 120 solves the optimization problem of the cost function that is defined using the sum of the sum Σ_f=1 ^FL_{MV_f}(w_f) of the cost function L_{MV_f}(w_f) and the regularization term ηΣ_f=1 ^F-1Σ_d=1 ^D|c_f,d−c_f+1,d|_c under a constraint condition (**).

(Case 1)

The optimization unit 120 will be described below with reference to FIG. 7 and FIG. 8 . FIG. 7 is a block diagram showing the configuration of the optimization unit 120. FIG. 8 is a flowchart showing the behavior of the optimization unit 120. As shown in FIG. 7 , the optimization unit 120 includes an initialization unit 121, a candidate calculation unit 122 and an optimum value determination unit 123.

The behavior of the optimization unit 120 will be described with FIG. 8 . Here, c_f=(c_f,1, . . . , c_f,D) (f=1, . . . , F) is a discrete variable that is defined by the discrete variable c_f,1, . . . , c_f,D.

In S121, the initialization unit 121 initializes α₀[c_f] (f=1, . . . , F), by the following expression.
α₀[c _f]=0  [Math. 25]

In S122, the candidate calculation unit 122 calculates α_f[c_f] for all values that the discrete variable c_f can have, for each the frequency f, and sets the value of the variable c_opt to c_opt=argmin_cα_F[c].

$\begin{array}{cc}\left[\mathrm{Math}.26\right]& \\ w_f^{\mathrm{dp}}\left[c_f\right]\leftarrow \underset{w_f}{\mathrm{argmin}}{L}_{{\mathrm{MV}}_f}\left(w_f\right)& \end{array}$ $s.t.\angle \left(w_f^H{a}_{f,d}\right)\in \left[2\pi \left(c_{f,d}-1\right)/C,2\pi c_{f,d}/C\right]$ $\left(d=1,\cdots ,D\right)w_f^H{a}_{f,1}=1$ $c_f^{\mathrm{prev}}\left[c_f\right]\leftarrow \underset{c_{f-1}^{\prime }}{\mathrm{argmin}}\left({\alpha }_{f-1}\left[c_{f-1}^{\prime }\right]+{\hat{L}}_{\eta f-1}\left(c_{f-1}^{\prime },c_f\right)\right)$ ${\alpha }_f\left[c_f\right]\leftarrow {\alpha }_{f-1}\left[c_f^{\mathrm{prev}}\left[c_f\right]\right]+{\hat{L}}_{\eta f-1}\left(c_f^{\mathrm{prev}}\left[c_f\right],c_f\right)+L_{{\mathrm{MV}}_f}\left(w_f^{\mathrm{dp}}\left[c_f\right]\right)$ ${(}^{\bigwedge }{L}_{\eta f}\left(c_f,c_{f+1}\right)=\eta {\sum }_{d=1}^D{❘c_{f,d}-c_{f+1,d}❘}_c)$

In S123, using the value of the variable c_opt calculated in S122 as an input, the optimum value determination unit 123 calculates the optimum value w_f* of the filter coefficient w_f and the value of the variable c_opt for the frequency bin f, in descending order from F to 1, by the following expression, and obtains the optimum value w* from w*={w₁*, . . . , w_F*}.
w _f *←w _f ^dp[c _opt]
c _opt ←c _f ^prev[c _opt]  [Math. 27]

(Case 2)

In this case, the optimization unit 120 may calculate the optimization value w* by solving an optimization problem min_{{c_f,w_f}}(Σ_f=1 ^FL_{MV_f}(w_f)+Σ_f=1 ^FΣ_d=1 ^D{circumflex over ( )}_{(f,d),c_f,d}(w_f ^Ha_f,d)+ηΣ_f=1 ^F-1Σ_d=1 ^D|c_f,d−c_f+1,d|_c) relevant to the filter coefficient w and the discrete variable c₁, . . . , c_F, instead of solving the optimization problem under the constraint condition (**). Here, c_f=(c_f,1, . . . , c_f,D) (f=1, . . . , F) is a discrete variable that is defined by the discrete variable c_f,1, . . . , c_f,D, and {circumflex over ( )}_{(f,d),c_f,d} (f=1, . . . , F, d=1, . . . , D) is a function relevant to a variable γ_f,d that is defined by the following expression (γ_f,d=w_f ^Ha_f,d).

$\begin{array}{cc}\left[\mathrm{Math}.28\right]& \\ {\Lambda }_{\left(f,d\right),c_{f,d}}\left({\gamma }_{f,d}\right)=\{\begin{array}{c}0(R\left({\gamma }_{f,d}{e}^{-2\pi j\left(c_{f,d}+1\right)/2C}\ge 1,\frac{2\pi c_{f,d}}{C}\le {\mathrm{\angle \gamma }}_{f,d}\le \frac{2\pi \left(c_{f,d}+1\right)}{C}\right)\\ \infty \left(\mathrm{otherwise}\right)\end{array}& \left(21\right)\end{array}$

The optimization unit 120 will be described below with reference to FIG. 7 and FIG. 8 . FIG. 7 is a block diagram showing the configuration of the optimization unit 120. FIG. 8 is a flowchart showing the behavior of the optimization unit 120. As shown in FIG. 7 , the optimization unit 120 includes an initialization unit 121, a candidate calculation unit 122 and an optimum value determination unit 123.

The behavior of the optimization unit 120 will be described with FIG. 8 .

In S121, the initialization unit 121 initializes α₀[c_f] (f=1, . . . , F), by the following expression.
α₀[c _f]=0  [Math. 29]

In S122, the candidate calculation unit 122 calculates α_f[c_f] for all values that the discrete variable c_f can have, for each the frequency bin f, and sets the value of the variable c_opt to c_opt=argmin_cα_F[c].

$\begin{array}{cc}\left[\mathrm{Math}.30\right]& \\ \begin{array}{cc}{w}_f^{\mathrm{dp}}\left[c_f\right]\leftarrow \underset{w_f}{\arg \min }\left(L_{{\mathrm{MV}}_f}\left(w_f\right)+\stackrel{D}{\sum _{d=1}}{\Lambda }_{\left(f,d\right),c_{f,d}}{w}_f^H{a}_{f,d}\right))& \end{array}& \end{array}$ $c_f^{\mathrm{prev}}\left[c_f\right]\leftarrow \underset{c_{f-1}^{\prime }}{\mathrm{argmin}}\left({\alpha }_{f-1}\left[c_{f-1}^{\prime }\right]+{\hat{L}}_{\eta f-1}\left(c_{f-1}^{\prime },c_f\right)\right)$ ${\alpha }_f\left[c_f\right]\leftarrow {\alpha }_{f-1}\left[c_f^{\mathrm{prev}}\left[c_f\right]\right]+{\hat{L}}_{\eta f-1}\left(c_f^{\mathrm{prev}}\left[c_f\right],c_f\right)+L_{{\mathrm{MV}}_f}\left(w_f^{\mathrm{dp}}\left[c_f\right]\right)+\stackrel{D}{\sum _{d=1}}{\Lambda }_{\left(f,d\right),c_{f,d}}\left({w_f^{\mathrm{dp}}\left[c_f\right]}^H{a}_{f,d}\right)$ ${(}^{\bigwedge }{L}_{\eta f}\left(c_f,c_{f+1}\right)=\eta {\sum }_{d=1}^D{❘c_{f,d}-c_{f+1,d}❘}_c)$

In S123, using the value of the variable c_opt calculated in S122 as an input, the optimum value determination unit 123 calculates the optimum value w_f* of the filter coefficient w_f and the value of the variable c_opt for the frequency bin f, in descending order from F to 1, by the following expression, and obtains the optimum value w* from w*={w₁*, . . . , w_F}.
w _f *←w _f ^dp[c _opt]
c _opt ←c _f ^prov[c _opt]  [Math. 31]

As described above, it can be said that the optimization unit 120 calculates the optimum value w* by performing derivation so as to reduce the difference in phase between adjacent frequency bins relevant to the response w_f ^Ha_f,d (f=1, . . . , F, d=1, . . . , D) of the beamformer in the frequency bin f for the angular direction θ_d.

According to the embodiments of the present invention, it is possible to design a stable beamformer having a good quality, by optimizing the filter coefficient in consideration of the relationship of the filter coefficient between adjacent frequency bins.

<Supplement>

FIG. 9 is a diagram showing an example of the functional configuration of a computer that realizes the apparatuses described above. The processing in the apparatuses described above can be executed when a recording unit 2020 reads programs for causing a computer to function as the apparatuses described above and a control unit 2010, an input unit 2030, an output unit 2040 and the like to behave.

For example, as a single hardware entity, the apparatus in the present invention includes an input unit that can be connected with a keyboard and the like, an output unit that can be connected with a liquid crystal display and the like, a communication unit that can be connected with a communication device (for example, a communication cable) capable of communicating with the exterior of the hardware entity, a CPU (Central Processing Unit, a cache memory, a register and the like may be included), a RAM and a ROM that are memories, an external storage device that is a hard disk, and a bus that connects the input unit, the output unit, the communication unit, the CPU, the RAM, the ROM and the external storage device such that data can be exchanged. Further, as necessary, the hardware entity may be provided with a device (drive) that can perform reading and writing for a record medium such as a CD-ROM. As a physical entity including the hardware resources, there are a general-purpose computer and the like.

In the external storage device of the hardware entity, programs necessary for realizing the above functions, data necessary in the processing of the programs, and the like are stored (for example, the program may be stored in a ROM that is a read-only storage without being limited to the external storage device). Further, data and others obtained by the processing of the programs are appropriately stored in the RAM, the external storage device or the like.

In the hardware entity, the programs stored in the external storage device (or the ROM or the like) and the data necessary for the processing of the programs are read in the memory as necessary, and are appropriately interpreted, executed or processed by the CPU. As a result, the CPU realizes predetermined functions (the above component units expressed as the . . . unit, the . . . means and the like).

The present invention is not limited to the above-described embodiments, and modifications can be appropriately made without departing from the spirit of the present invention. Further, the processes described in the above embodiments do not need to be executed in a time-series manner in the order of the descriptions, and may be executed in parallel or individually, depending on the processing capacities of the devices that execute the processes or as necessary.

In the case where the processing functions in the hardware entity (the apparatus in the present invention) described in the above embodiments are realized by a computer as described above, the processing contents of the functions to be included in the hardware entity are described by programs. Then, the programs are executed by the computer, and thereby, the processing functions in the above hardware entity are realized on the computer.

The programs describing the processing contents can be recorded in a computer-readable record medium. As the computer-readable record medium, for example, a magnetic record device, an optical disk, a magneto-optical record medium, a semiconductor memory and others may be used. Specifically, for example, a hard disk device, a flexible disk, a magnetic tape or the like can be used as the magnetic record device, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only Memory), a CD-R (Readable)/RW (ReWritable) or the like can be used as the optical disk, an MO (Magneto-Optical disc) or the like can be used as the magneto-optical record medium, and an EEP-ROM (Electronically Erasable and Programmable-Read Only Memory) or the like can be used as the semiconductor memory.

For example, the distribution of the programs is performed by sale, transfer, lending or the like of a portable record medium such as a DVD or CD-ROM in which the programs are recorded. Furthermore, the programs may be distributed by storing the programs in a storage device of a server computer and transmitting the programs from the server computer to another computer through a network.

For example, the computer that executes the programs, first, once stores the programs recorded in the portable record medium or the programs transmitted from the server computer, in its own storage device. Then, at the time of the execution of the processing, the computer reads a program stored in its own storage device, and executes a process in accordance with the read program. Further, as another form of the execution of the programs, the computer may read a program directly from the portable record medium, and may execute a process in accordance with the program.

Furthermore, whenever a program is transmitted from the server computer to the computer, the computer may execute a process in accordance with the received program. Further, the above-described processes may be executed by a so-called ASP (Application Service Provider) service in which the processing functions are realized by only execution instruction and result acquisition, without the transmission of the programs from the server computer to the computer. The program in the form includes information that is supplied for the processing by an electronic computer and that is similar to the program (for example, data that is not a direct command to the computer but has a property of prescribing the processing by the computer).

In the form, the hardware entity is configured by executing predetermined programs on the computer, but at least some of the processing contents may be realized in hardware.

The above description of the embodiment of the present invention has been presented for the purpose of exemplification and description. It is not intended to be exhaustive, and it is not intended to limit the invention to the disclosed strict form. Modifications and variations can be made from the above disclosure. The embodiments are selected and expressed, such that the best exemplification of the principle of the present invention is provided and such that a person skilled in the art can use the present invention as various embodiments suitable for deliberated actual use or can use the present invention while adding various modifications. All modifications and variations fall within the scope of the present invention that is determined by the attached claims interpreted based on a range given justly, lawfully and fairly.

Claims (13)

The invention claimed is:

1. A filter coefficient optimization apparatus including an optimization unit that calculates an optimum value w* of a filter coefficient w={w₁, . . . , W_F} (w_f(f=1, . . . , F, F is an integer equal to or more than 1) is a filter coefficient of a frequency bin f) of a beamformer that emphasizes sound (hereinafter referred to as target sound) from D sound sources (hereinafter referred to as a sound source 1, . . . , a sound source D),

D being an integer equal to or more than 1,

the optimization unit calculating the optimum value w* based on an optimization problem min_wL(w) relevant to the filter coefficient w, under a predetermined constraint condition,

L(w) being a cost function relevant to the filter coefficient w and being defined using a sum of a sum Σ_f=1 ^FL_{MV_f}(w_f) of a cost function L_{MV_f}(w_f) and a predetermined regularization term,

L_{MV_f}(w_f)=w_f ^HR_fw_f(f=1, . . . , F) being a cost function relevant to the filter coefficient w_f.

R_f(f=1, . . . , F) being a spatial correlation matrix for sound other than the target sound relevant to the frequency bin f,

the predetermined regularization term being defined using a difference in phase between adjacent frequency bins relevant to a response w_f ^Ha_{f,d (f=}1, . . . , F, d=1, . . . , D) of the beamformer in the frequency bin f for an angular direction θ_d.

θ_d(d=1, . . . , D) being the angular direction in which a sound source d exists, and

a_f,d(f=1, . . . , F, d=1, . . . , D) being an array manifold vector in the frequency bin f corresponding to a sound wave that comes from the angular direction θ_d, the sound wave being a plane wave; and

an output unit that sets the optimum value w* to the frequency bin of the beamformer and emphasizes at least a part of the sound wave as the target sound.

2. The filter coefficient optimization apparatus according to claim 1, wherein

the predetermined regularization term is ηΣ_f=1 ^F−1Σ_d=1 ^D|∠(w_f ^Ha_f,d)−∠(w_f+1 ^Ha_f+1,d)|_2πor ηΣ_f=1 ^F−2Σ_d=1 ^D|∠(w_f ^Ha_f,d)−2∠(w_f+1 ^Ha_f+1 ^Ha_f+1,d)+∠(w_f+2 ^Ha_f+2,d)|_2π.

η being a predetermined positive number, and

∠(w_f ^Ha_f,d) (f=1, . . . , F, d=1, . . . , D) being the phase of the response w_f ^Ha_f,d of the beamformer in the frequency bin f for the angular direction θ_d.

3. The filter coefficient optimization apparatus according to claim 1, wherein:

the predetermined regularization term is

ηΣ_f=1 ^F−1Σ_d=1 ^D|c_f,d-c_f+1,d|_c.

η being a predetermined positive number,

C being an integer equal to or more than 1,

∠(w_f ^Ha_f,d) (f=1, . . . , F, d=1, . . . , D) being the phase of the response w_f ^Ha_f,d of the beamformer in the frequency bin f for the angular direction θ_d, and

c_f,d(f=1, . . . , F, d=1, . . . , D) being a discrete variable having one value of 1, . . . , C that satisfies ∠(w_f ^Ha_f,d)∈[2π(c_f,d−1)/C, 2πc_f,d/C] for the phase ∠(w_f ^Ha_f,d).

4. The filter coefficient optimization apparatus according to claim 3, wherein the predetermined constraint condition is expressed by the following expression:

w _f ^Hα_f,d=1  [Math. 32]

(f=1, . . . , F).

5. The filter coefficient optimization apparatus according to claim 3, wherein the predetermined constraint condition is expressed by the following expression:

w _f ^Hα_f,d≥1  [Math. 33]

(f=1, . . . , F, d=1, . . . , D).

6. The filter coefficient optimization apparatus according to claim 4, wherein

the optimization unit includes

a candidate calculation unit that calculates α_f[c_f] for all values that a discrete variable c_f can have, for each frequency bin f, by the following expression, and adopts c_opt=argmin_cα_F[c] as a value of a variable c_opt,

c_f=(c_f,1, . . . , c_f,D) (f=1, . . . , F) being the discrete variable that is defined by a discrete variable c_f,1, . . . , c_f,D.

$\begin{array}{cc}\left[\mathrm{Math}.34\right]& \\ w_f^{\mathrm{dp}}\left[c_f\right]\leftarrow \underset{w_f}{\mathrm{argmin}}{L}_{{\mathrm{MV}}_f}\left(w_f\right)& \end{array}$ $s.t.\angle \left(w_f^H{a}_{f,d}\right)$ $\in \left[2\pi \left(c_{f,d}-1\right)/C,2\pi c_{f,d}/C\right]$ $\left(d=1,\dots ,D\right),w_f^H{a}_{f,1}=1$ $c_f^{\mathrm{prev}}\left[c_f\right]\leftarrow \underset{c_{f-1}^{\prime }}{\arg \min }\left({\alpha }_{f-1}\left[c_{f-1}^{\prime }\right]+{\hat{L}}_{\eta f-1}\left(c_{f-1}^{\prime },c_f\right)\right)$ ${\alpha }_f\left[c_f\right]\leftarrow {\alpha }_{f-1}\left[c_f^{\mathrm{prev}}\left[c_f\right]\right]+{\hat{L}}_{\eta f-1}\left(c_f^{\mathrm{prev}}\left[c_f\right],c_f\right)+L_{{\mathrm{MV}}_f}\left(w_f^{\mathrm{dp}}\left[c_f\right]\right)$ ${(}^{\bigwedge }{L}_{\eta f}\left(c_f,c_{f+1}\right)=\eta {\sum }_{d=1}^D{❘c_{f,d}-c_{f+1,d}❘}_c),\mathrm{and}$

an optimum value determination unit that calculates an optimum value w_f* of the filter coefficient w_f and the value of the variable c_opt for the frequency bin f, in descending order from F to 1, by the following expression, and obtains the optimum value w* from w*={w₁*, . . . , w_F*}:

w _f *←w _f ^dp[c _opt]

c _opt ←c _f ^prev[c _opt].  [Math. 35]

7. The filter coefficient optimization apparatus according to claim 5, wherein:

${\Lambda }_{\left(f,d\right),c_{f,d}}\left({\gamma }_{f,d}\right)=\{\begin{array}{c}0\left(R\left({\gamma }_{f,d}{e}^{-2\pi j\left(c_{f,d}+1\right)/2C}\right)\ge 1,\frac{2\pi c_{f,d}}{C}\le {\mathrm{\angle \gamma }}_{f,d}\le \frac{2\pi \left(c_{f,d}+1\right)}{C}\right)\\ \infty \left(\mathrm{otherwise}\right)\end{array}$

the optimization unit calculates the optimum value w* by solving an optimization problem min_{{c_f,w_f}}(Σ_f=1 ^FL_{MV_f}(w_f)+Σ_f=1 ^FΣ _d=1 ^D{circumflex over ( )}_{(f,d)c_f,d}(w_f ^Ha_f,d)+ηΣ_f=1 ^F−1Σ_d=1 ^D|d_f,d-c_f+1,3|c) relevant to the filter coefficient w and a discrete variable c₁, . . . , c_F, instead of solving the optimization problem min_wL(w),

c_f=(c_f,1, . . . , c_f,D) (f=1. . . , F) being the discrete variable that is defined by the discrete variable c_f,1, . . . , c_f,D, and

{circumflex over ( )}_{(f,d),c_f,d}(f=1, . . . , F, d=1, . . . , D) is a function relevant to a variable γ_f,d that is defined by the following expression (γ_f,d=w_f ^Ha_f,d):

$\begin{array}{cc}\left[\mathrm{Math}.36\right]& \\ {\Lambda }_{\left(f,d\right),c_{f,d}}\left({\gamma }_{f,d}\right)=\{\begin{array}{c}0\left(R\left({\gamma }_{f,d}{e}^{-2\pi j\left(c_{f,d}+1\right)/2C}\right)\ge 1,\frac{2\pi c_{f,d}}{C}\le {\mathrm{\angle \gamma }}_{f,d}\le \frac{2\pi \left(c_{f,d}+1\right)}{C}\right)\\ \infty \left(\mathrm{otherwise}\right)\end{array}.& \end{array}$

8. The filter coefficient optimization apparatus according to claim 7, wherein

the optimization unit includes

a candidate calculation unit that calculates α_f[c_f] for all values that the discrete variable _f can have, for each frequency bin f, by the following expression, and adopts as a C_opt=argmin_cα_F[c] value of a variable c_opt:

$\left[\mathrm{Math}.37\right]$ $w_f^{\mathrm{dp}}\left[c_f\right]\leftarrow \underset{w_f}{\arg \min }\left(L_{{\mathrm{MV}}_f}\left(w_f\right)+\stackrel{D}{\sum _{d=1}}{\Lambda }_{\left(f,d\right),c_{f,d}}\left(w_f^H{a}_{f,d}\right)\right)$ $c_f^{\mathrm{prev}}\left[c_f\right]\leftarrow \underset{c_{f-1}^{\prime }}{\mathrm{argmin}}\left({\alpha }_{f-1}\left[c_{f-1}^{\prime }\right]+{\hat{L}}_{\eta f-1}\left(c_{f-1}^{\prime },c_f\right)\right)$ ${\alpha }_f\left[c_f\right]\leftarrow {\alpha }_{f-1}\left[c_f^{\mathrm{prev}}\left[c_f\right]\right]+{\hat{L}}_{\eta f-1}\left(c_f^{\mathrm{prev}}\left[c_f\right],c_f\right)+$ $L_{{\mathrm{MV}}_f}\left(w_f^{\mathrm{dp}}\left[c_f\right]\right)+\stackrel{D}{\sum _{d=1}}{\Lambda }_{\left(f,d\right),c_{f,d}}\left({w_f^{\mathrm{dp}}\left[c_f\right]}^H{a}_{f,d}\right)$ ${(}^{\bigwedge }{L}_{\eta f}\left(c_f,c_{f+1}\right)=\eta {\sum }_{d=1}^D{❘c_{f,d}-c_{f+1,d}❘}_c)\mathrm{and}$

an optimum value determination unit that calculates an optimum value w_f* of the filter coefficient w_f and the value of the variable Copt for the frequency bin f, in descending order from F to 1, by the following expression, and obtains the optimum value w* from w*={w₁*, . . . ,w_F*}:

w _f *←w _f ^dp[c _opt]

c _opt ←c _f ^prev[c _opt].  [Math. 38]

9. A filter coefficient optimization apparatus including an optimization unit that calculates an optimum value w* of a filter coefficient w={w₁, . . . , w_F} (w_f(f=1, . . . , F, F is an integer equal to or more than 1) is a filter coefficient of a frequency bin f) of a beamformer that emphasizes sound (hereinafter referred to as target sound) from D sound sources (hereinafter referred to as a sound source 1, . . . , a sound source D),

D being an integer equal to or more than 1,

the optimization unit calculating the optimum value w* by performing derivation so as to reduce a difference in phase between adjacent frequency bins relevant to a response w_f ^Ha_f,d(f=1, . . . , , F, d=1, . . . , D) of the beamformer in the frequency bin f for an angular direction θ_d,

θ_d(d=1, . . . , D) being the angular direction in which a sound source d exists,

a_f,d(f=1, . . . , F, d=1, . . . , D) being an array manifold vector in the frequency bin f corresponding to a sound wave that comes from the angular direction θ_d, the sound wave being a plane wave; and

an output unit that sets the optimum value w* to the frequency bin of the beamformer and emphasizes at least a part of the sound wave as the target sound.

10. A filter coefficient optimization method including an optimization step in which a filter coefficient optimization apparatus calculates an optimum value w* of a filter coefficient w={w₁, . . . , w_F} (w_f(f=1, F, F is an integer equal to or more than 1) is a filter coefficient of a frequency bin f) of a beamformer that emphasizes sound (hereinafter referred to as target sound) from D sound sources (hereinafter referred to as a sound source 1, . . . , a sound source D),

D being an integer equal to or more than 1,

the optimization step being a step of calculating the optimum value w* based on an optimization problem min_wL(w) relevant to the filter coefficient w, under a predetermined constraint condition,

L(w) being a cost function relevant to the filter coefficient w and being defined using a sum of a sum Σ_f=1 ^Fl_{MV_f}(w_f) of a cost function L_{MV_f}(w_f) and a predetermined regularization term,

L_{MV_f}(w_f)=w_f ^HR_fw_f(f=1, . . . , F) being a cost function relevant to the filter coefficient w_f,

R_f(f=1, . . . , F) being a spatial correlation matrix for sound other than the target sound relevant to the frequency bin f,

the predetermined regularization term being defined using a difference in phase between adjacent frequency bins relevant to a response w_f ^Ha_f,d(f=1, . . . , F, d=1, . . . , D) of the beamformer in the frequency bin f for an angular direction θ_d,

θ_d(d=1, . . . , D) being the angular direction in which a sound source d exists, and

a_f,d(f=1, . . . , F, d=1, . . . , D) being an array manifold vector in the frequency bin f corresponding to a sound wave that comes from the angular direction θ_d, the sound wave being a plane wave; and

an output step that sets the optimum value w* to the frequency bin of the beamformer and emphasizes at least a part of the sound wave as the target sound.

11. A filter coefficient optimization method including an optimization step in which a filter coefficient optimization apparatus calculates an optimum value w* of a filter coefficient w={w₁, . . . , w_F} (w_f(f=1, . . . , F, F is an integer equal to or more than 1) is a filter coefficient of a frequency bin f) of a beamformer that emphasizes sound (hereinafter referred to as target sound) from D sound sources (hereinafter referred to as a sound source 1, . . . , a sound source D),

D being an integer equal to or more than 1,

the optimization step being a step of calculating the optimum value w* by performing derivation so as to reduce a difference in phase between adjacent frequency bins relevant to a response w_f ^Ha_f,d(f=1, . . . , F, d=1, . . . , D) of the beamformer in the frequency bin f for an angular direction θ_d,

θ_d(d=1, . . . ,D) being the angular direction in which a sound source d exists, and

a_f,d(f=1, . . . , F, d=1, . . . , D) being an array manifold vector in the frequency bin f corresponding to a sound wave that comes from the angular direction θ_d, the sound wave being a plane wave; and

an output step that sets the optimum value w* to the frequency bin of the beamformer and emphasizes at least a part of the sound wave as the target sound.

12. A non-transitory computer-readable recording medium storing a program that causes a computer to function as the filter coefficient optimization apparatus according to claim 1.

13. A non-transitory computer-readable recording medium storing a program that causes a computer to function as the filter coefficient optimization apparatus according to claim 9.

Images