JP2018028580A - Sound source enhancement learning device, sound source enhancement device, sound source enhancement learning method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a sound source enhancement learning device for learning sound source enhancement in which sound quality degradation is suppressed.SOLUTION: A sound source enhancement learning device includes: a Wiener filter templating unit 110 for a finite number of Wiener filter templates from a set of frequency domain target sound learning data and frequency domain noise learning data; an action-value function initialization unit 120 for initializing an action-value function; a sound source enhancement unit 130 for applying an appropriate Wiener filter template selected on the basis of a state vector generated from the set of frequency domain target sound learning data and frequency domain noise learning data and a value of the action-value function calculated using Wiener filter template data to a frequency domain observation signal and thereby generating an enhanced target sound; an action-value function update unit 150 for updating the action-value function using an auditory evaluation value calculated from the enhanced target sound; and a convergence determination unit 160 for outputting the action-value function when a prescribed convergence condition is satisfied.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a sound source enhancement technique, and more particularly to a sound source enhancement technique using a Wiener filter learned using reinforcement learning.

  In information processing technology using sound such as voice recognition and sports broadcasting, it is necessary to clearly collect a specific desired sound (hereinafter referred to as a target sound) using a microphone. However, when the sound is emphasized with the current microphone, ambient noise is collected together with the target sound. Due to the influence of the noise collected together, there is a problem that in speech recognition, speech is buried in noise and speech recognition becomes difficult. Also, there is a problem that the sports sound is drowned out in cheers and the sense of realism is not transmitted in sports broadcasting.

ここで、t∈{1, …, T}とω∈{1, …, Ω}は、それぞれ時間と周波数のインデックスである。 As a technique for solving such a problem, there is a sound source enhancement technique that clearly emphasizes only a target sound. Sound source enhancement emphasizes only the target sound S _{ω, t} from the observed signal X _{ω, t} collected by the microphone, which is a mixture of the target sound S _{ω, t} and the noise N _{ω, t to} be emphasized at time t (See equation (1)).

Here, t∈ {1,..., T} and ω∈ {1,..., Ω} are time and frequency indexes, respectively.

In the sound source enhancement using the Wiener filter, the target sound S _{ω, t} and the noise N _{ω, t} are assumed to be uncorrelated and the signal (emphasis target sound) Y _ω that emphasizes the target sound S _{ω, t} by the following equation _{, t} .

That is, in the sound source enhancement by the Wiener filter, it is important to design the Wiener filter _{Gω, t} accurately from the observation signal _{Xω, t} .

In recent years, it has been found that the accuracy of the Wiener filter design is improved by using a statistical machine learning technique (Non-Patent Document 1). In the Wiener filter design based on statistical machine learning, a function M (X _{ω, t} ) for predicting an ideal Wiener filter ^ G _{ω, t} is learned from learning data.

具体的な最適化手続きでは、式(5)や式(6)の二乗誤差を関数M(・)のパラメータで偏微分し勾配法で最小化することにより、関数M(・)を学習する。 First, the learning data S _{ω, 1,..., Ttrain of} the target sound and the learning data N _{ω, 1,.} Next, pseudo observation sound _{Xω, 1, ..., Ttrain} and ideal Wiener filter _{Gω, 1, ..., Ttrain} are designed based on Equation (1) and Equation (3). Then, a function M (•) for converting X _{ω, t} to ^ G _{ω, t} is expressed by a neural network or the like, and a pseudo observed sound X _{ω, 1, ..., Ttrain} and an ideal Wiener filter G _ω, Learn using a set of _{1, ..., Ttrain} . At this time, a square error objective function is often used as a learning criterion for the function M (•). For example, Equation (5) based on the Wiener filter error or Equation (6) based on the error of the target sound is used.

In a specific optimization procedure, the function M (•) is learned by partially differentiating the square error of the equations (5) and (6) with the parameters of the function M (•) and minimizing by the gradient method.

  However, for example, in the Wiener filter designed using the function M (・) learned on the basis of Equation (5), the target sound is emphasized, but the sound quality deteriorates due to the influence of nonlinear distortion and the like. Arise. This is because Equation (5) is based only on the closeness of the spectrum of the target sound and the output sound, and does not consider the characteristics of human hearing.

  By the way, PESQ (Non-Patent Document 2), STOI (Non-Patent Document 3), PEASS (Non-Patent Document 4), and the like are known as performance indexes for sound source enhancement in consideration of human auditory characteristics and the like. Hereinafter, these performance indexes are collectively referred to as auditory ratings. This auditory rating score indicates that the higher the value, the more perceived that the sound quality is acceptable for human beings.

  If statistical sound source emphasis can be learned so that the auditory rating having such a property becomes large, it is possible to design a Wiener filter that does not deteriorate the sound quality.

Y. Xu, J. Du, LR Dai and CH Lee, “A regression approach to speech enhancement based on deep neural networks”, IEEE / ACM Trans.Audio, Speech and Language Processing, Vol.23, No.1, pp. 7-19, 2015. AW Rix, JG Beerends, MP Hollier and AP Hekstra, “Perceptual evaluation of speech quality (PESQ): A new method for speech quality assessment of telephone networks and codecs”, in Proc. ICASSP '01, pp.749-752, 2001 . CH Taal, RC Hendriks, R. Heusdens, and J. Jensen, “An Algorithm for Intelligibility Prediction of Time-Frequency Weighted Noisy Speech”, IEEE Trans. Audio, Speech and Language Processing, Vol. 19, No. 7, pp. 2125-2136, 2011. Valentin Emiya, Emmanuel Vincent, Niklas Harlander and Volker Hohmann, “Subjective and objective quality assessment of audio source separation”, IEEE Trans.Audio, Speech and Language Processing, Vol.19, No.7, pp.2046-2057, 2011.

However, there are two problems in learning statistical sound source enhancement so as to maximize the auditory rating.
(1) The auditory score cannot be calculated by a simple calculation formula such as Expression (5) or Expression (6), but is calculated using a complicated calculation expression. Therefore, it is difficult to obtain the derivative, and the auditory rating cannot be maximized directly using a gradient method or the like.
(2) The auditory score cannot be calculated unless one sound source data (for example, one utterance for speech) is completed, instead of being obtained for each frame as in Equation (5) and Equation (6). .

  For this reason, it has been difficult to realize statistical sound source enhancement learning that increases the auditory score. In other words, it has been difficult to design a Wiener filter that does not deteriorate the sound quality.

  Therefore, an object of the present invention is to provide a sound source enhancement learning apparatus that performs sound source enhancement learning while suppressing sound quality deterioration using reinforcement learning.

  One aspect of the present invention includes a winner filter template generation unit that generates a finite number of winner filters as a winner filter template from a set of frequency domain target sound learning data and frequency domain noise learning data, and an action value that initializes an action value function. A function initialization unit; and a state vector expressed using a frequency domain observation signal generated from a set of the frequency domain target sound learning data and the frequency domain noise learning data, and the state vector and the Wiener filter template A sound source emphasizing unit that generates an emphasized target sound by applying an optimal Wiener filter template selected based on the value of the behavior value function calculated using the frequency domain observation signal, and calculating from the emphasized target sound Action value function for updating the action value function using the obtained auditory score Including a Shinbu, and a convergence determination unit which outputs the action value function if it meets a predetermined convergence condition.

  One aspect of the present invention includes a winner filter template generation unit that generates a winner filter that satisfies the first criterion from a set of frequency domain target sound learning data and frequency domain noise learning data as a winner filter template, and an initial value of an action value function Action value function initialization unit for generating a state vector expressed using a frequency domain observation signal generated from a set of the frequency domain target sound learning data and the frequency domain noise learning data, and the state vector And applying the optimal winner filter template selected based on the value of the action value function calculated using the winner filter template to the frequency domain observation signal, and a sound source emphasizing unit that generates an enhanced target sound, Using the auditory rating score, which is a value obtained by evaluating the emphasis target sound, An action value function updating unit that updates the action value function so that a winner filter template that satisfies the criterion 2 is selected, and a convergence determination unit that outputs the action value function when a predetermined convergence condition is satisfied. Including.

  According to the present invention, it is possible to perform sound source enhancement learning while suppressing deterioration in sound quality by using reinforcement learning.

The figure explaining the concept of reinforcement learning using a game. The figure explaining the weighting for suppressing a hearing score fall. FIG. 2 is a block diagram showing a configuration of a sound source enhancement learning device 100. 5 is a flowchart showing the operation of the sound source enhancement learning apparatus 100. The block diagram which shows the structure of the winner filter template-izing part 110. FIG. The flowchart which shows operation | movement of the winner filter template-izing part 110. FIG. The block diagram which shows the structure of the sound source emphasis part. 5 is a flowchart showing the operation of the sound source emphasizing unit 130. The block diagram which shows the structure of the sound source emphasis learning apparatus. The flowchart which shows operation | movement of the sound source emphasis learning apparatus 200. The block diagram which shows the structure of the auditory score binarization part 240. FIG. The flowchart which shows operation | movement of the auditory score binarization part 240. FIG. The block diagram which shows the structure of the sound source emphasis learning apparatus 300. FIG. The flowchart which shows operation | movement of the sound source emphasis learning apparatus 300. The block diagram which shows the structure of the update weight calculation part 340. FIG. The flowchart which shows operation | movement of the update weight calculation part 340. The block diagram which shows the structure of the sound source emphasis learning apparatus. 5 is a flowchart showing the operation of the sound source enhancement learning apparatus 400. The block diagram which shows the structure of the action value function initialization part 420. FIG. The flowchart which shows operation | movement of the action value function initialization part 420. FIG. 3 is a block diagram showing a configuration of a sound source enhancement apparatus 500. 5 is a flowchart showing the operation of the sound source emphasizing apparatus 500.

  Hereinafter, embodiments of the present invention will be described in detail. In addition, the same number is attached | subjected to the structure part which has the same function, and duplication description is abbreviate | omitted.

で決定する。
(3)環境は、エージェントの行動a_tによって状態x_t+1に変化する。
(4)環境は、状態x_t+1に基づき行動a_tの報酬r_tをエージェントに返す。
(5)(1)へ戻る。 First, reinforcement learning will be described.
<Reinforcement Learning>
Reinforcement learning is a type of machine learning that deals with problems in which an agent in a certain environment observes the current state and decides an action. At time t, the agent determines one action a∈ {1,..., A} from A types of actions based on the observation from the environment (that is, the current state of the environment) x _t . The action a _{t at} time t is determined based on the value of the action value function Q (x, a). In summary, the flow is as follows. Note that the observation (state) x is generally expressed as a vector.
(1) agents, at time t, receives the observation x _t from the environment.
(2) The agent action value function Q (x, a) determined to perform an optimal action a _t at time t based on. In general,

To decide.
(3) environment is changed to the state x _{t + 1} by the agent of the action a _t.
(4) environment, returns a reward r _t of action a _t based on the state x _{t + 1} to the agent.
(5) Return to (1).

  FIG. 1 illustrates the flow of this process using a game. First, the agent determines how to move the lever according to the game state indicated by the current game screen. According to the decision, the agent operates the lever. Then, the game screen changes and the score is updated.

  The agent operates the lever so that the score at the end of the game is increased. Game practice is to play the game many times, and to obtain a criterion (action value function Q (x, a)) for how to move the lever on what game screen to increase the final score That is.

  The action value function Q (x, a) can be easily expressed in the form of a table function if the number of states x and the number of action patterns A that the observed state x can take is small. However, in the problem of information processing for actual sounds and images, the observed state x usually has a huge number of states because it takes continuous values such as sound pressure and pixel values.

とした上で、

や

と表現される。ここで、Lはネットワークの層の数、W^(j)とb^(j)はそれぞれ第j層の重み行列とバイアスベクトル（j∈{1, …,L}）、F(・)はシグモイド関数などの非線形変換を表す。また、u_t ^(j)は第j層における出力（ベクトル）である。u_t ^(j)(a)はベクトルu_t ^(j)のa番目の要素を表す。 Therefore, the action value function Q (x, a) is often expressed by being approximated by some other function. For example, the action value function based on deep neural network (DNN) is

And then

And

It is expressed. Where L is the number of network layers, W ^(j) and b ^(j) are the weight matrix and bias vector (j∈ {1,…, L}) of the jth layer, and F (·) is the sigmoid function. Represents a non-linear transformation. U _t ^(j) is an output (vector) in the j-th layer. u _t ^(j) (a) represents the a-th element of the vector u _t ^(j) .

ただし、γ（0≦γ<1）は、R_tが有限の値となるように設定する割引率である。 Learning in reinforcement learning is a problem of learning an action value function Q (x, a) so that an optimum action can always be selected. The optimal action value function is a function that gives a policy that maximizes the sum R _{t of} rewards r that can be obtained from the present time t defined by Equation (13) to an infinite future.

However, γ (0 ≦ γ <1) is a discount rate set so that R _t becomes a finite value.

  Hereinafter, “behavior value function is optimal” is used in the sense that it is the behavior value function that maximizes the value of Equation (13).

を最小化することで学習することができる。これにより、例えば、勾配法を用いて行動価値関数Q(x, a|Θ)を学習することができる。つまり、

によりパラメータΘを更新していき、行動価値関数Q(x, a|Θ)を求めるとよい。ここで、αは正の実数であり、∇はナブラを表す。 If an optimal action value function Q ^opt (x, a) exists, the action value function Q (x, a | Θ) determined by the parameter Θ is

Can be learned by minimizing. Thereby, for example, the behavior value function Q (x, a | Θ) can be learned using the gradient method. That means

The parameter Θ is updated according to the above, and the action value function Q (x, a | Θ) is obtained. Here, α is a positive real number, and ∇ represents Nabula.

However, since the optimal action value function Q ^opt (x, a) is unknown, Equation (14) cannot be calculated as it is unless approximated in some form.

Therefore, in the experience replay algorithm, an action is determined and executed according to the current action value function Q (x, a | Θ), and the parameter Θ is updated according to the reward r _t obtained therefrom. In the previous game example, the game is played many times and the lever operation policy is improved based on the result.

と設定する。そして、目標値との誤差を表す関数（以下、誤差関数ともいう）L_Θを

として計算し、式(15)を用いてパラメータΘを更新する。 First, the current action-value function Q (x, a | Θ) in accordance with, the observation x _{1, ..., T,} action a _{1, ..., T,} reward r _{1, ...,} to get a pair of _T. Then, the action value function Q _t ^target as a ^target (ie, approximating Q ^opt (x, a))

And set. A function (hereinafter also referred to as an error function) L _Θ representing an error from the target value is

And the parameter Θ is updated using equation (15).

However, the experience-replay algorithm, when determining the action a _t at time t always Equation (7), there is a problem that behavior to be selected will be biased in dependence on the initial value.

Therefore, in the ε-greedy algorithm, at each time t, an action a _t is randomly selected with a probability ε. By doing so, it is possible to prevent the bias of the selected action and to learn a more optimal action value function Q (x, a | Θ).

Next, the principle of the invention will be described.
<Principle of the invention>
The basic framework for reinforcement learning of statistical sound source enhancement that improves auditory ratings is as follows. That is, learning of the action value function Q (x, a | Θ) is advanced according to the procedures (1) to (4).
(1) An observation signal X _{ω, t} at the current time t of the environment, an agent as a function M (•), and a reward as an auditory rating Z _iter (where iter is an index indicating repetition). Therefore, the winner filter (number identifying the winner filter) determined according to the function M (•) corresponds to the action.
(2) Using the target sound learning data S _{ω, 1, ..., Ttrain} and the noise learning data N _{ω, 1, ..., Ttrain} , the observation signal X _{ω, 1, ..., Ttrain} which is a pseudo observation sound , And perform sound source enhancement according to the action value function Q (x, a | Θ) of the current function M (•) (state x is a variable expressed using observation signals X _{ω, t} , action a is Number to identify the winner filter). However, the function M (•), which is an estimation function of the Wiener filter, must be changed to a form realized by several behavior patterns.
(3) The auditory rating score Z _iter is calculated from the result of sound source enhancement.
(4) The action value function Q (x, a | Θ) is updated so as to maximize the auditory rating score Z _iter as a reward.

  However, the following four tasks remain in this learning procedure.

  Problem 1 is a problem that must be solved, and the above procedure cannot be executed unless this problem is solved. On the other hand, the tasks 2 to 4 are optional, and the learning accuracy of the sound source enhancement is further improved by solving these tasks.

(Problem 1) The function M (•) is a function that returns a continuous value Wiener filter. Therefore, there are an infinite number of patterns that the value of the function M (•) can take. However, in reinforcement learning, the action pattern of the function M (•), which is an agent, must be dropped into a finite number A. That is, it is necessary to make the Wiener filter used in this learning finite.

(Problem 2) The auditory score takes a continuous value, not a binary value as in the case of winning or losing a game. In this case, the behavioral value function Q (x, a | Θ) is a regression type function that estimates the auditory score directly as shown in Equation (12) (for example, the projection from real number to real number such as multiple regression analysis) It is common to design with a function. However, if the behavior value function Q (x, a | Θ) is expressed by a regression type function, the solution space is generally widened, so that learning becomes difficult.

  To solve this problem, binarize the auditory score and convert the behavioral value function Q (x, a | Θ) from real number to binary as shown in Equation (11), such as logistic regression. Projective function).

(Problem 3) The auditory rating cannot be evaluated without emphasizing one utterance. In addition, the auditory score has a property that the sound quality deteriorates only in a certain local part of one utterance and the score is lowered even if perfect emphasis is performed in the other part. However, even if the agent receives the auditory score itself, it does not know the reason why the auditory score has been lowered, and therefore, the part that has been completely emphasized is determined to have acted badly (see FIG. 2).

  In order to solve this problem, an algorithm that corrects only a portion (frame) causing a decrease in the auditory rating score is required.

(Problem 4) In reinforcement learning, the optimal target value of the action value function Q (x, a | Θ) is not given. Specifically, the target value is sequentially given by equation (16). For this reason, learning tends to depend on the initial value and tends to fall into a local solution.

  In order to solve this problem, a method for determining an initial value suitable for sound source enhancement (that is, an initialization method for the action value function Q (x, a | Θ)) is required.

Hereinafter, a method for solving the above four problems will be described.
(Solution for Problem 1: Creating a template for the winner filter)
The Wiener filter G _{ω, t} is designed by Equation (3). Since the numerator of Equation (3) is determined by the target sound S _{ω, t} , it can be patterned to some extent depending on the nature of the target sound. For example, it is considered that a Wiener filter that emphasizes vowels will appear frequently if the target sound S _{ω, t} is a speech, and a Wiener filter that emphasizes sudden sounds such as kick sounds will appear frequently if the target sound S _{ω, t} is a speech. In other words, the Wiener filter G _{ω, t} can be sufficiently expressed by using several types of templates, and the action value function Q (x, a | Θ) selects which template (corresponding to the action) at the time t. Is a function that determines

Therefore, in order to solve the problem 1, it is only necessary to generate A winner filter templates from the target sound learning data S _{ω, 1,..., Ttrain} and the noise learning data N _{ω, 1,.} . Specifically, first, the learning data S _{ω, 1, ..., Ttrain of the} target sound and the noise learning data N _{ω, 1, ..., Ttrain} are used to obtain an ideal winner filter G _{ω, 1, …, Generate Ttrain} . Next, the Wiener filters G _{ω, 1,..., Ttrain} are clustered using K-means clustering, a histogram method, or the like to generate _A winner filter templates G _{ω, 1 ,.} Here, the template is a cluster center when K-means clustering, GMM (Gaussian Mixture Model), or vector quantization clustering is used, and each histogram bin when using the histogram method.

(Solution for Problem 2: Threshold judgment of auditory score)
In reinforce learning, an identification model can be applied to a reward that uses only two values, so that the action value function can be estimated with high accuracy. Therefore, the threshold evaluation is used to change the auditory score to a binarized auditory score that is binarized.

と決定する。 _Let Z _{iter be} the auditory score in the iter update, and use the threshold φ for the binary auditory score r _t ^iter that is the reward at the time t of the iter update.

And decide.

The threshold value φ is updated by the following procedure every ITER _thres-update .
(1) The average value of the auditory rating score Z _iter for ITER _thres-update is φ ⁻ .
(2) When the average value φ ⁻ is larger than φ−β, φ ← φ ⁻ + β is set (that is, otherwise, the threshold φ is not changed). Here, β is a positive real number indicating a bias to the threshold value.

ここで、Υはバイナリ化聴感評点r_t ^iterの取りうる値の最大値がRとなるように制御する正の実数である。Υの値は用いる聴感評点Zの種類によって決定すべきであるが、例えば、0<Z<5の場合はΥ=50程度に設定すればよい。 However, if the ^binarized auditory score r _t ^iter is determined using equation (19), the magnitude of the auditory rating score Z _iter is not considered at all. That is, the same ^binarized auditory rating r _t ^iter = R is obtained both when Z _iter = φ + 0.01 and when Z _iter = φ + 1000. Therefore, when the learning did not proceed well by using equation (19), the binarization audibility score r _t ^iter may be set as follows.

Here, Υ is a positive real number that is controlled so that the maximum value that can be taken by the binarized auditory score r _t ^iter is R. The value of Υ should be determined according to the type of auditory rating score Z to be used. For example, when 0 <Z <5, Υ = 50 may be set.

  In the above description, if the portion that is larger is larger, the portion that is larger is larger than the above, the portion that is smaller is smaller, the portion that is smaller is appropriately changed as follows. May be.

(Solution of Problem 3: Calculation of update weight based on square error)
A device for correcting only a part that causes a decrease in the auditory rating in one utterance will be described (see FIG. 2). Although the sound quality cannot be maximized only by the square error, the square error between the target sound Y _{ω, t} and the target sound S _{ω, t} is an index indicating the enhancement of the target sound. Here, an algorithm for adjusting the update amount of the parameter Θ according to the square error value for each frame t is proposed. A function L _Θ representing an error from the target value is calculated using the update weight w _t calculated for each frame t. This will be specifically described below.

なお、二乗誤差E_tは、メルフィルタバンクなどで圧縮したスペクトルから計算してもよい。 First, the square error E _t for each frame t of the emphasized target sound Y _{ω, t} and the target sound S _{ω, t} is calculated by the following equation (21).

Note that square error E _t may be calculated from the spectrum obtained by compressing the like mel filter bank.

Next, normalized by the following equation (22) the squared error E _t. The normalized square error _{Et is referred} to as a normalized square error.

つまり、聴感評点がよい場合は、行動価値関数Q(x, a|Θ)が大きくなるように更新する必要がある。そこで、正規化二乗誤差の小さなフレームを積極的に更新する。また、聴感評点が悪い場合は、行動価値関数Q(x, a|Θ)が小さくなるように更新する必要がある。そこで、正規化二乗誤差の大きなフレームを積極的に更新する。たとえて言うならば、前者はほめるイメージであり、後者は叱るイメージである。 Next, the update weight w _t is calculated by the following equation (23).

That is, when the auditory rating is good, it is necessary to update the behavior value function Q (x, a | Θ) to be large. Therefore, a frame with a small normalized square error is actively updated. If the auditory rating is bad, it is necessary to update the behavior value function Q (x, a | Θ) to be small. Therefore, a frame with a large normalized square error is actively updated. For example, the former is an image of compliment, and the latter is an image of scolding.

Next, a function L _Θ representing an error from the target value is calculated by the following equation (24). That is, Expression (24) replaces Expression (18).

Finally, the parameter Θ is updated according to the equation (15) using this L _Θ .

Here has been described how using a square error in the calculation of E _t seek updating weight w _t, the error used to calculate the E _t is not limited to the square error. Since it is only necessary to calculate the degree of distortion of the signals of the enhanced target sound Y _{ω, t} and the target sound S _{ω, t} for each frame t, for example, instead of the square error, a signal-to-distortion ratio SDR (Signal-to-Distortion Ratio) Similar effects can be obtained by using a signal-to-interference ratio (SIR) or a weighted sum thereof. Therefore, the square error and the normalized square error may be simply referred to as an error and a normalization error, respectively.

(Solution for Problem 4: Determination of square error based initial value)
Here, an initialization method of the behavior value function Q (x, a | Θ) will be described. For example, when the behavior value function Q (x, a | Θ) is expressed using equations (8) to (11), the initial values of W ^(j) and b ^(j) will be determined. . As described above _, the square error between the emphasized target sound Y _{ω, t} and the target sound S _{ω, t} is an index indicating the enhancement of the target sound. Therefore, the initial value is an action value function Q (x, a | Θ) that outputs an optimal Wiener filter number in the sense of a square error in each frame of learning data. Hereinafter, the procedure will be described.

First, the ideal Wiener filter G _{ω, 1, ..., Ttrain} is generated from the learning data S _{ω, 1, ..., Ttrain of} the target sound and the learning data N _{ω, 1, ..., Ttrain of the noise} by Equation (3) . _{Here, S ω, 1, ...,} Ttrain and N _{ω, 1, ...,} observation signals X _omega from _{Ttrain, 1, ..., Ttrain} be kept generated.

ここで、aは（課題１の解決法：ウィナーフィルタのテンプレート化）で述べたテンプレートを識別する番号であり、a∈{1, …, A}である。 Next, in each frame of the learning data (t = 1,..., Ttrain), the optimum Wiener filter template number a _{1,..., Ttrain} is determined using the following equation (25) in terms of a square error.

Here, a is a number for identifying the template described in (Solution to Problem 1: Template of Wiener filter), and a∈ {1,..., A}.

  The square error in equation (25) may be calculated from a Wiener filter compressed by a mel filter bank or the like.

Finally, the observed signal X _{ω, 1, ...,} action-value function Q (x, a | Θ) so as to output the number a _t of the template when you learn the state vector x _t that is generated from _Ttrain the identification learning To do. In the previous example, the initial values of W ^(j) and b ^(j) are determined. An arbitrary method can be used for identification learning. For example, logistic regression may be used.

  As a specific method for generating the state vector, a method similar to the method for generating the state vector in the sound source emphasizing unit 130 of the first embodiment to be described later may be used.

<Embodiment 1>
Hereinafter, the sound source enhancement learning apparatus 100 according to the first embodiment will be described with reference to FIGS. FIG. 3 is a block diagram illustrating a configuration of the sound source enhancement learning device 100. FIG. 4 is a flowchart showing the operation of the sound source enhancement learning apparatus 100. As shown in FIG. 1, the sound source enhancement learning device 100 includes a winner filter template conversion unit 110, a behavior value function initialization unit 120, a sound source enhancement unit 130, an auditory score calculation unit 140, and a behavior value function update unit 150. The convergence determination unit 160 is included.

  The sound source enhancement learning device 100 is connected to the target sound learning data recording unit 910 and the noise learning data recording unit 920. In the target sound learning data recording unit 910 and the noise learning data recording unit 920, the target sound and noise collected in advance are recorded as learning data. The target sound should be a clean sound that does not contain any noise.

  The target sound and noise recorded in the target sound learning data recording unit 910 and the noise learning data recording unit 920 are preferably time domain signals. The time domain target sound and time domain noise are recorded separately for each sound source. For example, when the target sound is speech, it is divided into speech units. In the following, for the sake of simplicity, even a target sound other than voice will be referred to as utterance.

  In the following description, it is assumed that the target sound and noise recorded in the target sound learning data recording unit 910 and the noise learning data recording unit 920 are time domain target sound and time domain noise divided into speech units.

  Various parameters (for example, parameters used in a learning algorithm such as reinforcement learning and identification learning) used in each component of the sound source enhancement learning device 100 may be input from the outside in the same manner as the target sound learning data and noise learning data. Alternatively, each component may be set in advance. The recommended values of the various parameters will be described as appropriate when explaining each component related to the parameters.

  As an example of the parameter, there is the number of behavior patterns (number of templates) A used in the winner filter templating unit 110. Since the value of the behavior pattern number A is preferably changed according to the number of utterances and the complexity of the target sound, it is better to input from the outside. In addition, the recommended value of the number A of behavior patterns is about 64 to 128 in the case of voice.

The winner filter template generation unit 110 receives the target sound learning data and the noise learning data as input, and generates _A winner filter templates _{Gω, 1,..., A} (S110). Specifically, a template is generated by the method described in (Solution for Problem 1: Template of Wiener Filter).

  Hereinafter, the winner filter templating unit 110 will be described with reference to FIGS. FIG. 5 is a block diagram illustrating a configuration of the winner filter template forming unit 110. FIG. 6 is a flowchart showing the operation of the winner filter template forming unit 110. As shown in FIG. 5, the Wiener filter template conversion unit 110 includes a frequency domain conversion unit 111, a Wiener filter generation unit 112, and a clustering unit 113.

First, the frequency domain conversion unit 111 converts the target sound learning data and the noise learning data read from the target sound learning data recording unit 910 and the noise learning data recording unit 920 into frequency domain target sound learning data S _{ω, 1,.} The frequency domain noise learning data N _{ω, 1,..., Ttrain} are converted (S111). For example, a time domain signal may be converted into a frequency domain signal using fast Fourier transform (FFT). The FFT length and shift length, which are parameters required for conversion, may be set to 512, the shift length, 256, etc. when the sampling rate is 16 kHz.

Next, the winner filter generator 112 uses the learning data S _{ω, 1,..., Ttrain} and N _{ω, 1,..., Ttrain} generated in S111 to use the winner filter G _{ω, 1 ,. Ttrain} is generated (S112).

Finally, the clustering unit 113 generates _A winner filter templates _{Gω, 1, ...,} A from the winner filters _{Gω, 1, ..., Ttrain} (S113). As long as a finite number of templates can be generated, a classification method other than clustering may be used.

The behavior value function initialization unit 120 initializes the behavior value function Q (x, a | Θ) (S120). That is, the initial value of the behavior value function Q (x, a | Θ) is generated. When the action value function is expressed using equations (8) to (11), determine the initial values of the weight matrix W ^(j) of the jth layer and the bias vector b ^(j) of the jth layer become. For example, the values of the elements of the weight matrix W ^(j) and the bias vector b ^(j) may be generated using random numbers. Alternatively, it may be generated using cross-entropy-based backpropagation.

  The sound source emphasizing unit 130 generates an emphasis target sound, a template number, and a state vector using the winner filter template generated in S110 and the current action value function Q (x, a | Θ) (S130). The process of S130 is repeatedly executed for each set of learning data.

  Hereinafter, the sound source emphasizing unit 130 will be described with reference to FIGS. FIG. 7 is a block diagram illustrating a configuration of the sound source emphasizing unit 130. FIG. 8 is a flowchart showing the operation of the sound source emphasizing unit 130. As shown in FIG. 7, the sound source enhancement unit 130 includes an observation signal generation unit 131, a frequency domain conversion unit 132, a state vector generation unit 133, a template selection unit 134, an enhancement target sound generation unit 135, and a time domain conversion. Unit 136 and an output generation unit 137.

  First, the observation signal generation unit 131 reads out target sound learning data and noise learning data recorded in the target sound learning data recording unit 910 and the noise learning data recording unit 920, and superimposes the target sound learning data and the noise learning data, A time domain observation signal is generated (S131).

Next, the frequency domain conversion unit 132 converts the observation signal generated in S131 into the frequency domain _, and generates a frequency domain observation signal _{Xω, t} (S132). Similar to the frequency domain transform unit 111, a time domain signal may be converted into a frequency domain signal using fast Fourier transform (FFT).

The state vector generation unit 133 generates a state vector x _t at each time t from the observation signal X _{ω, t} generated in S132 (S133). For example, the state vector xt may be generated by vertically linking observation signals from the past P ₁ frame to the future P ₂ frame of the frame _t as shown in the following equation (26).

Note that a signal compressed by a mel filter bank or the like may be used as an observation signal to be connected. Further, P ₁ and P ₂ may be set to about 10.

The template selection unit 134 uses the winner filter template G _{ω, 1,..., A} generated in S110 to use the action value function Q (x _t , a | Θ) (where x _t is the state vector generated in S133, a is a template number), and an optimum winner filter template (template number a _t ) is selected using equation (7) (S134). The value of the behavior value function Q (x _t , a | Θ) may be calculated using, for example, Expression (8) to Expression (11).

  Note that a Wiener filter template may be selected using the ε-greedy algorithm instead of using Equation (7). In this case, the value of ε may be set to 0.01 or 0.05.

ただし、a_tは、Ｓ１３４で選択した最適なウィナーフィルタテンプレートを識別する番号であり、a_t∈{1, …, A}である。 The enhancement target sound generation unit 135 generates the frequency domain enhancement target sound Y _{ω, t} using the optimum winner filter template G _{ω, at} selected in S134 and Equation (27) (S135).

However, a _t is a number that identifies the optimum Wiener filter template selected in _{S134, a t ∈ {1,} ..., A} is.

The time domain conversion unit 136 generates a time domain enhancement target sound from the enhancement target sound Y _{ω, t} generated in S135 (S136). An inverse Fourier transform may be used for the conversion to the time domain.

Finally, the output generation unit 137 outputs the time domain emphasis target sound, the selected optimal winner filter template number a ₁ ,..., And the state vectors x ₁ ,.

Note that the sound source emphasizing unit 130 may not include the time domain conversion unit 136. In this case, S136 is omitted, and the output in S137 is the frequency domain emphasis target sound, the selected optimal winner filter template number a ₁ ,..., And the state vector x ₁ ,.

The audibility score calculator 140 calculates the audibility score Z _iter from the emphasized target sound output in S130 (S140). As the auditory rating as an index of sound quality, any one such as PESQ and STOI can be used. It should be noted that the target sound learning data and the noise learning data read from the target sound learning data recording unit 910 and the noise learning data recording unit 920 are read if necessary in calculating these auditory scores. In addition, when the enhancement target sound output in S130 is a frequency domain signal, the auditory rating score Z _iter is calculated after being converted to a time domain signal as necessary.

The behavior value function updating unit 150 updates the behavior value function Q (x, a | Θ) using the auditory rating score calculated in S140 (S150). Specifically, the behavioral value function Q (x, a | Θ) is updated by using the auditory rating score Z _iter as a reward and updating the parameter Θ using Equation (18) and Equation (15). Each value _{Q (x t, a t |} Θ) action value function when calculating the template number a ₁ output in S130, ..., the state vector x ₁ at each time is calculated ... with.

In the equation (15), α may be set to about 10 ⁻³ .

  Convergence determination unit 160 outputs the current action value function Q (x, a | Θ) when the number of updates reaches the predetermined number specified at the start of execution, and ends the process, but has not reached Returns to S130 and performs update calculation of the action value function Q (x, a | Θ) again (S160).

  According to the invention of the present embodiment, by applying reinforcement learning to sound source enhancement, it is possible to use an objective function other than the square error represented by Equation (5) and Equation (6). As a result, sound source enhancement can be optimized using an objective function suitable for sound information processing technology (specifically, an action value function reflecting an auditory rating score). That is, it is possible to perform sound source enhancement learning with suppressed sound quality deterioration.

<Embodiment 2>
In the first embodiment, the auditory score calculated by the auditory score calculator 140 is generally a continuous value. However, as described in (Solution for Problem 2: Threshold evaluation of auditory score), in reinforcement learning, it is possible to apply a discrimination model when using a binary value as a reward. Can do. Therefore, a process of binarizing the auditory rating score, which is a continuous value, using threshold determination is added.

  Hereinafter, the sound source enhancement learning apparatus 200 according to the second embodiment will be described with reference to FIGS. 9 to 10. FIG. 9 is a block diagram illustrating a configuration of the sound source enhancement learning device 200. FIG. 10 is a flowchart showing the operation of the sound source enhancement learning device 200. The sound source enhancement learning device 200 is different from the sound source enhancement learning device 100 only in that an auditory score binarization unit 240 is added.

  Therefore, hereinafter, the auditory score binarization unit 240 will be described with reference to FIGS. FIG. 11 is a block diagram showing the configuration of the auditory score binarization unit 240. FIG. 12 is a flowchart showing the operation of the auditory score binarization unit 240. As shown in FIG. 11, the auditory score binarization unit 240 includes a binarization unit 241 and a threshold update unit 242.

The binarization unit 241 performs binary conversion on the auditory rating score Z _iter generated in S140 to generate a binarized auditory score r _t ^iter (S241). Specifically, Equation (19) of (Solution of Problem 2: Threshold evaluation of auditory rating score) is used. That is, when the auditory rating score Z _iter is equal to or greater than the threshold φ, the binarized auditory score r _t ^iter is R, and when the auditory score Z _iter is smaller than the threshold φ, the binarized auditory score r _t ^iter is −R.

Note that the ^binarized auditory score r _t ^iter may be generated using the equation (20) instead of the equation (19).

The threshold update unit 242 updates the threshold φ every time S241 is executed ITER _thres-update times (S242). Specific method is: a street in (Problem 2 solutions threshold determination audibility score), the average value of the perceptual score Z _iter phi ^- only if greater than φ-β, φ ← φ ^- updated by + beta To do.

R may be set to about 0.05, β may be set to about 3, initial value of φ may be set to 0, and ITER _thres-update may be set to about 20.

  The behavior value function updating unit 150 updates the behavior value function Q (x, a | Θ) using the binarized auditory score calculated in S240 as a reward (S150).

  According to the invention of the present embodiment, since a binary value is used as an auditory rating score in reinforcement learning, an action value function can be estimated with high accuracy. That is, it is possible to perform sound source enhancement learning that further suppresses sound quality degradation.

<Embodiment 3>
In the first embodiment, the action value function updating unit 150 calculates a function L _Θ representing an error from the target value using Expression (18). However, as described in (Solution to Problem 3: Calculation of update weight based on square error), frame t is corrected so that only the part that causes a decrease in auditory rating in one utterance is corrected. The action value function can be estimated more accurately by calculating the square error value of L _Θ every time. Therefore, a process for calculating the update weight w _t used when calculating the function L _Θ is added.

  Hereinafter, the sound source enhancement learning apparatus 300 according to the third embodiment will be described with reference to FIGS. 13 to 14. FIG. 13 is a block diagram illustrating a configuration of the sound source enhancement learning device 300. FIG. 14 is a flowchart showing the operation of the sound source enhancement learning device 300. The sound source enhancement learning device 300 is different from the sound source enhancement learning device 100 in that an update weight calculation unit 340 is added and an action value function update unit 350 is added instead of the action value function update unit 150.

  Hereinafter, first, the update weight calculation unit 340 will be described with reference to FIGS. 15 to 16. FIG. 15 is a block diagram illustrating a configuration of the update weight calculation unit 340. FIG. 16 is a flowchart showing the operation of the update weight calculation unit 340. As shown in FIG. 15, the update weight calculation unit 340 includes a frequency domain conversion unit 341, an error calculation unit 342, an error normalization unit 343, and an update weight determination unit 344.

The frequency domain conversion unit 341 converts the target sound learning data recorded in the target sound learning data recording unit 910 into the frequency domain _, and generates the frequency domain target sound S _{ω, t} (S341). Similar to the frequency domain transform unit 111, a time domain signal may be converted into a frequency domain signal using fast Fourier transform (FFT).

Error calculator 342, a target sound S _omega generated in _{S341, t} and frequency domain emphasis target sound generated in S135 Y _omega, calculates the square error E _t using equation (21) from _t (S342). Error normalization unit 343 calculates a normalized squared error using the equation (22) from the square error E _t (S343).

The update weight determination unit 344 determines an update weight from the normalized square error (S344). A specific method uses Equation (23). That is, if the auditory rating score Z _iter is equal to or greater than the threshold φ, the update weight w _t is a value obtained by subtracting the normalized square error from 1. If the auditory rating score Z _iter is smaller than the threshold φ, the update weight w _{t is set} to the normalized square error. Let it be.

  In S342 to S344, the description has been given using the square error and the normalized square error. However, as described in (Solution of Problem 3: Calculation of update weight based on square error), an error other than the square error is used. May be. In this case, the square error and the normalized square error in S342 to S344 may be read as an error and a normalization error.

Next, the behavior value function updating unit 350 will be described. The behavior value function updating unit 350 updates the behavior value function Q (x, a | Θ) using the auditory rating score calculated in S140 and the update weight calculated in S340 (S350). Specifically, the auditory score Z _iter is used as a reward, L _Θ is calculated using Equation (24) instead of Equation (18), and the parameter Θ is updated according to Equation (15). Update (x, a | Θ).

According to the invention of this embodiment, in order to calculate the square error value in L _{Θ for} each frame t so as to correct only the part that causes a decrease in auditory rating in one utterance, The value function can be estimated with high accuracy. That is, it is possible to perform sound source enhancement learning that further suppresses sound quality degradation.

<Embodiment 4>
In the first embodiment, since the action value function is calculated sequentially using the equation (16), there is a problem that reinforcement learning tends to depend on the initial value and easily falls into a local solution. Therefore, the process of the behavior value function initialization unit is changed so that the calculation of the behavior value function can be started from the initial value close to the optimum target value. As a result, the behavior value function can be accurately estimated.

  Hereinafter, the sound source enhancement learning apparatus 400 according to the fourth embodiment will be described with reference to FIGS. 17 to 18. FIG. 17 is a block diagram illustrating a configuration of the sound source enhancement learning apparatus 400. FIG. 18 is a flowchart showing the operation of the sound source enhancement learning apparatus 400. The sound source enhancement learning device 400 differs from the sound source enhancement learning device 100 only in that a behavior value function initialization unit 420 is added instead of the behavior value function initialization unit 120.

  Therefore, hereinafter, the behavior value function initialization unit 420 will be described with reference to FIGS. 19 to 20. FIG. 19 is a block diagram showing a configuration of the behavior value function initialization unit 420. FIG. 20 is a flowchart showing the operation of the behavior value function initialization unit 420. As shown in FIG. 19, the behavior value function initialization unit 420 includes an optimum winner filter determination unit 421, an observation signal generation unit 131, a frequency domain conversion unit 132, a state vector generation unit 133, and a behavior value function identification learning unit. 422.

The optimum winner filter determination unit 421 determines the winner filter numbers a _{1,..., Ttrain} using the winner filters G _{ω, 1,..., Ttrain} generated in S112 (S421). Specifically, Expression (25) of (Solution of Problem 4: Determination of initial value based on square error) is used. That is, the optimum Wiener filter number a _{1,..., Ttrain} is determined in _terms of the square error.

  The observation signal generation unit 131, the frequency domain conversion unit 132, and the state vector generation unit 133 generate state vectors from the target sound learning data and the noise learning data read from the target sound learning data recording unit 910 and the noise learning data recording unit 920. (S131-S133).

Action value function identifying and learning unit 422, the state vector x _t and action value function Q from a set of Wiener filter corresponding to the determined Wiener filter number a _t at S421 (x, a | Θ) identifying learning (S422). Specifically, the state vector x action value function as _t and outputs the number a _t templates when you enter Q (x, a | Θ) may be an identification learning.

  According to the invention of this embodiment, since the learning of the behavior value function starts from a more preferable initial value in reinforcement learning, the behavior value function can be estimated with high accuracy. That is, it is possible to perform sound source enhancement learning that further suppresses sound quality degradation.

<Embodiment 5>
In the first to fourth embodiments, an action value function Q (x, a | Θ) (where x is a state vector generated from a set of target sound learning data and noise learning data, (a represents the template number of the Wiener filter). Here, a method for generating an emphasized target sound from an observation signal collected by a microphone using the action value function Q (x, a | Θ) learned in the first to fourth embodiments will be described. As a result, it is possible to output an emphasized target sound in which the target sound in the observation signal is emphasized as a sound source with suppressed sound quality deterioration.

  The action value function Q (x, a | Θ) here is expressed using the value of Θ at the end of learning.

  Hereinafter, a sound source emphasizing apparatus 500 according to the fifth embodiment will be described with reference to FIGS. FIG. 21 is a block diagram illustrating a configuration of the sound source enhancement device 500. FIG. 21 is a flowchart showing the operation of the sound source emphasizing apparatus 500. As illustrated in FIG. 21, the sound source enhancement device 500 includes a state vector generation unit 510, an action value function evaluation unit 520, and an enhancement target sound generation unit 530.

  The sound source emphasizing apparatus 500 is connected to the observation signal recording unit 930. In the observation signal recording unit 930, an observation signal collected in advance is recorded. The observation signal is a target of sound source enhancement, and is assumed to be recorded as a frequency domain signal for simplicity.

Furthermore, the sound source emphasizing apparatus 500 is connected to the learning result recording unit 940. In the learning result recording unit 940, a winner filter template _{Gω, 1,..., A} and an action value function Q (x, a | Θ) generated using any of the sound source enhancement learning devices 100 to 400 in advance are recorded. Has been.

The state vector generation unit 510 uses the observation signal X _{ω, t} (where t∈ {1,..., T}, ω∈ {1,..., Ω}) collected by the microphone to obtain the state using Equation (26). A vector x _t is generated (S510).

The behavior value function evaluation unit 520 reads the winner filter template G _{ω, 1,..., A} and the behavior value function Q (x, a | Θ) read from the learning result recording unit 940, and each winner filter template G _{ω, 1 ,..., A is} calculated as an action value function Q (x _t , a | Θ) (where x _t is the state vector generated in S510, and a is a template number indicating 1 to A). Is used to select the optimal winner filter template G _{ω, at} (template number a _t ) (S520). That is, basically the same processing as that performed by the template selection unit 134 is executed.

The emphasis target sound generation unit 530 generates the frequency domain emphasis target sound Y _{ω, t} using the optimal Wiener filter template G _{ω, at} selected in S520 and Equation (27) (S530). That is, basically, the same processing as that of the enhancement target sound generation unit 135 is executed.

  According to the invention of the present embodiment, it is possible to output an emphasized target sound in which the target sound in the observation signal is emphasized as a sound source with suppressed sound quality deterioration.

<Modification 1>
In the first to fourth embodiments, the auditory score is adopted as the reward, but a reward other than the auditory score can be used. For example, when it is desired to optimize sound source enhancement for speech recognition, a binary value indicating whether speech recognition is correct or incorrect may be used as a reward. In this case, the auditory score calculation unit 140 in the first to fourth embodiments may be replaced with a speech recognition unit that outputs a result of speech recognition using the emphasized target sound output in S130 as an input.

<Modification 2>
The winner filter template forming unit 110 according to the first to fourth embodiments generates a finite number of winner filter templates using clustering. Instead of using clustering, a Wiener filter is generated based on a square error such as Equation (5) or Equation (6), and the SN ratio as in Equation (3) is used as a reference (hereinafter referred to as the first reference). A Wiener filter template having a high SN ratio may be generated. In addition, the winner filter template may be designed in consideration of the characteristics of the input sound so that the SN ratio is high with respect to a specific input sound (for example, voice or sports sound).

  When the Wiener filter template is generated in this way, the auditory rating does not necessarily increase. Therefore, the behavior value function updating unit 150 updates the behavior value function so that the auditory score becomes high with the auditory score as a reference (hereinafter referred to as a second reference). Specifically, not only the first reference, The action value function is updated by selecting a winner filter template that also satisfies the second criterion.

  This makes it possible to perform sound source enhancement learning with suppressed sound quality degradation.

<Supplementary note>
The apparatus of the present invention includes, for example, a single hardware entity as an input unit to which a keyboard or the like can be connected, an output unit to which a liquid crystal display or the like can be connected, and a communication device (for example, a communication cable) capable of communicating outside the hardware entity. Can be connected to a communication unit, a CPU (Central Processing Unit, may include a cache memory or a register), a RAM or ROM that is a memory, an external storage device that is a hard disk, and an input unit, an output unit, or a communication unit thereof , A CPU, a RAM, a ROM, and a bus connected so that data can be exchanged between the external storage devices. If necessary, the hardware entity may be provided with a device (drive) that can read and write a recording medium such as a CD-ROM. A physical entity having such hardware resources includes a general-purpose computer.

  The external storage device of the hardware entity stores a program necessary for realizing the above functions and data necessary for processing the program (not limited to the external storage device, for example, reading a program) It may be stored in a ROM that is a dedicated storage device). Data obtained by the processing of these programs is appropriately stored in a RAM or an external storage device.

  In the hardware entity, each program stored in an external storage device (or ROM or the like) and data necessary for processing each program are read into a memory as necessary, and are interpreted and executed by a CPU as appropriate. . As a result, the CPU realizes a predetermined function (respective component requirements expressed as the above-described unit, unit, etc.).

  The present invention is not limited to the above-described embodiment, and can be appropriately changed without departing from the spirit of the present invention. In addition, the processing described in the above embodiment may be executed not only in time series according to the order of description but also in parallel or individually as required by the processing capability of the apparatus that executes the processing. .

  As described above, when the processing functions in the hardware entity (the apparatus of the present invention) described in the above embodiments are realized by a computer, the processing contents of the functions that the hardware entity should have are described by a program. Then, by executing this program on a computer, the processing functions in the hardware entity are realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like, and as an optical disk, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only). Memory), CD-R (Recordable) / RW (ReWritable), etc., magneto-optical recording medium, MO (Magneto-Optical disc), etc., semiconductor memory, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. Can be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. When executing the process, the computer reads a program stored in its own recording medium and executes a process according to the read program. As another execution form of the program, the computer may directly read the program from a portable recording medium and execute processing according to the program, and the program is transferred from the server computer to the computer. Each time, the processing according to the received program may be executed sequentially. Also, the program is not transferred from the server computer to the computer, and the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by the execution instruction and result acquisition. It is good. Note that the program in this embodiment includes information that is used for processing by an electronic computer and that conforms to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer).

  In this embodiment, a hardware entity is configured by executing a predetermined program on a computer. However, at least a part of these processing contents may be realized by hardware.

DESCRIPTION OF SYMBOLS 100 Sound source emphasis learning apparatus 110 Wiener filter template conversion part 111 Frequency domain conversion part 112 Wiener filter generation part 113 Clustering part 120 Action value function initialization part 130 Sound source emphasis part 131 Observation signal generation part 132 Frequency domain conversion part 133 State vector generation part 134 Template selection unit 135 Enhancement target sound generation unit 136 Time domain conversion unit 137 Output generation unit 140 Auditory score calculation unit 150 Action value function update unit 160 Convergence determination unit 200 Sound source enhancement learning device 240 Auditory score binarization unit 241 Binary conversion unit 242 Threshold update unit 300 Sound source enhancement learning device 340 Update weight calculation unit 341 Frequency domain conversion unit 342 Error calculation unit 343 Error normalization unit 344 Update weight determination unit 400 Sound source enhancement learning device 420 Action value function initialization unit 421 Optimal winner The filter determining unit 422 action value function identification learning unit 500 sound enhancement apparatus 510 state vector generator 520 action value function evaluation section 530 emphasis target sound generator

Claims (8)

A winner filter templating unit that generates a finite number of winner filters as a winner filter template from a set of frequency domain target sound learning data and frequency domain noise learning data;
An action value function initialization unit for generating an initial value of the action value function;
A state vector expressed using a frequency domain observation signal generated from a set of the frequency domain target sound learning data and the frequency domain noise learning data is generated, and the state vector and the Wiener filter template are used to calculate the state vector. A sound source emphasizing unit that generates an emphasis target sound by applying an optimal Wiener filter template selected based on the value of the action value function to the frequency domain observation signal;
An action value function updating unit for updating the action value function using an auditory rating calculated from the emphasized target sound;
A sound source enhancement learning device comprising: a convergence determination unit that outputs the action value function when a predetermined convergence condition is satisfied. The sound source enhancement learning device according to claim 1,
further,
A sound source enhancement learning apparatus including an auditory score binarization unit that generates an auditory score obtained by binary conversion of the auditory score. The sound source enhancement learning device according to claim 1 or 2,
A function that is used when the behavior value function update unit updates the behavior value function, and is expressed as a sum of values calculated for each frame using the auditory score (hereinafter referred to as an error value) for all frames. Let the error function L _Θ be
further,
An update weight calculation unit that calculates an update weight for each frame using the frequency domain target sound learning data and the frequency domain emphasized target sound generated by the sound source enhancement unit;
The behavior value function updating unit uses the error function L _Θ as the function expressed as the sum of the error function L _Θ multiplied by the update weight and the value for all frames. A sound source enhancement learning apparatus that updates The sound source enhancement learning device according to any one of claims 1 to 3,
The behavior value function initialization unit includes:
A number for identifying an optimal winner filter is determined from among the winner filters generated from the set of the frequency domain target sound learning data and the frequency domain noise learning data, and the frequency domain target sound learning data and the frequency domain noise learning data are determined. A sound source emphasizing learning apparatus using an action value function discriminated and learned using a state vector generated from a set of the number and the number set as the initial value. A sound source that generates a frequency domain emphasized target sound obtained by sound source emphasizing a frequency domain observation signal, using the winner filter template and the action value function generated by using the sound source emphasis learning device according to claim 1. An emphasis device,
A state vector generator for generating a state vector from the frequency domain observation signal;
An action value function evaluation unit that selects an optimal winner filter template based on the value of the action value function calculated using the state vector and the winner filter template;
A sound source emphasizing apparatus including: an emphasis target sound generation unit that generates the frequency domain emphasis target sound from the frequency domain observation signal using the optimal winner filter template. A Wiener filter templating unit that generates a Wiener filter that satisfies the first criterion from a set of frequency domain target sound learning data and frequency domain noise learning data as a Wiener filter template;
An action value function initialization unit for generating an initial value of the action value function;
A state vector expressed using a frequency domain observation signal generated from a set of the frequency domain target sound learning data and the frequency domain noise learning data is generated, and the state vector and the Wiener filter template are used to calculate the state vector. A sound source emphasizing unit that generates an emphasis target sound by applying an optimal Wiener filter template selected based on the value of the action value function to the frequency domain observation signal;
An action value function update unit that updates the action value function so that a winner filter template that satisfies the first criterion and the second criterion is selected by using an auditory score that is a value obtained by evaluating the emphasized target sound. When,
A sound source enhancement learning device comprising: a convergence determination unit that outputs the action value function when a predetermined convergence condition is satisfied. A sound source enhancement learning method in which a sound source enhancement learning device generates and outputs an action value function from a set of frequency domain target sound learning data and frequency domain noise learning data,
The sound source enhancement learning device generates a finite number of winner filters from the set of the frequency domain target sound learning data and the frequency domain noise learning data as a winner filter template,
The sound source emphasis learning device initializes the action value function, an action value function initialization step;
The sound source enhancement learning device generates a state vector expressed using a frequency domain observation signal generated from a set of the frequency domain target sound learning data and the frequency domain noise learning data, and the state vector and the Wiener filter A sound source emphasizing step for generating an emphasized target sound by applying an optimal Wiener filter template selected based on the value of the action value function calculated using a template to the frequency domain observation signal;
An action value function update step in which the sound source enhancement learning device updates the action value function using an auditory score calculated from the emphasized target sound;
A sound source enhancement learning method comprising: a convergence determination step of outputting the action value function when the sound source enhancement learning device satisfies a predetermined convergence condition.   A program for causing a computer to function as the sound source enhancement learning device according to any one of claims 1 to 4 or the sound source enhancement device according to claim 5.

Images