JP2018028618A - Parameter estimation device for mask estimation, parameter estimation method for mask estimation, and parameter estimation program for mask estimation - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve the accuracy of speech recognition with limited learning data when performing speech recognition while adapting to a conventional environment.SOLUTION: A time frequency analysis unit 11 applies short time signal analysis to each of a plurality of observation signals to extract an observation signal for each time frequency point in a situation where an acoustic signal of a target sound source and an acoustic signal of noise coexist, and constitutes an observation vector for each time frequency point. A mask estimation unit 12 estimate a mask with regard to the acoustic signal on the basis of the observation vector and a parameter for mask estimation. A sound emphasis unit 13 multiplies the observation vector and the mask with regard to the acoustic signal of the target sound source together and thereby acquires an enhanced sound. A speech recognition unit 14 estimates the posterior probability of phoneme state of the enhanced sound using a parameter for speech recognition. A parameter estimation unit 15 estimates a parameter for mask estimation so that a distance reference between the posterior probability of phoneme state and a reference label is minimized.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a mask estimation parameter estimation apparatus, a mask estimation parameter estimation method, and a mask estimation parameter estimation program.

  Conventionally, as a method of performing speech recognition while adapting to the environment, a method of performing speech enhancement as a pre-processing of speech recognition (see, for example, Non-Patent Document 1), or a method of adapting the speech recognizer itself to the environment (eg, non-patent) Reference 2) is known.

N. Ito, S. Araki, T. Yoshioka, and T. Nakatani, "Relaxed disjointness based clustering for joint blind source separation and dereverberation," in Proc. Int. Worksh. Acoust. Echo, Noise Contr., Pp. 268- 272, 2014. D. Yu and L. Deng, "Automatic speech recognition: a deep learning approach," Springer, 2015. T. Yoshioka, N. Ito, M. Delcroix, A. Ogawa, K. Kinoshita, M. Fujimoto, C. Yu, WJ Fabian, M. Espi, T. Higuchi, S. Araki, and T. Nakatani, "The NTT CHiME-3 system: advances in speech enhancement and recognition for mobile multi-microphone devices, "in Proc.Worksh. Automat.Speech Recognition, Understanding, 2015, pp.436-443.

  However, the conventional method of performing speech recognition while adapting to the environment has a problem that a large amount of learning data may be required to improve the accuracy of speech recognition. For example, in the method described in Non-Patent Document 2, the accuracy of speech recognition is improved by adapting the parameters of a speech recognizer to the environment. Here, since the speech recognizer calculates phoneme state posterior probabilities by performing linear operations and nonlinear operations multiple times, it is composed of many parameters, so in order to adapt all parameters to the environment, a large amount Learning data may be required.

  The parameter estimation apparatus for mask estimation according to the present invention includes one first acoustic signal corresponding to a target sound source and N−1 second acoustic signals corresponding to noise (where N is an integer of 2 or more). In a situation where N acoustic signals including, are mixed, a short time signal analysis is applied to each of the M observation signals (where M is an integer of 2 or more) recorded at different positions. Extracting an observation signal for each frequency point, and based on the observation frequency and a parameter for mask estimation, which constitutes an observation vector that is an M-dimensional vertical vector of the observation signal for each time frequency point, A mask estimator for estimating a mask indicating how much each of the N acoustic signals is included in the observation vector for each of the time frequency points, the observation vector, and the first Using the speech enhancement unit that acquires the enhanced speech by multiplying the mask for the reverberation signal at each of the time frequency points, and the parameters for speech recognition learned in advance using learning data, A predetermined distance criterion between a speech recognition unit that estimates a phoneme state posterior probability that indicates which phoneme state the speech seems to be at each time, and a reference label of the phoneme state input from the outside And a parameter estimation unit that estimates the parameter for mask estimation so that is minimized.

  The mask estimation parameter estimation method of the present invention is a mask estimation parameter estimation method executed by a mask estimation parameter estimation device, and includes a first acoustic signal corresponding to a target sound source and noise. In a situation where N acoustic signals including the corresponding N-1 second acoustic signals are mixed, short-time signal analysis is applied to each of M observation signals recorded at different positions. Based on the time-frequency analysis step of extracting an observation signal for each time-frequency point and constructing an observation vector that is an M-dimensional vertical vector of the observation signal for each time-frequency point, and the observation vector and a parameter for mask estimation A mask estimation for estimating a mask indicating how much each of the N acoustic signals is included in the observation vector for each time frequency point Then, the observation vector and the mask for the first acoustic signal are multiplied at each of the time frequency points, and a speech enhancement step for acquiring enhanced speech, and learning is performed in advance using learning data. A speech recognition step for estimating a phoneme state posterior probability that indicates which phoneme state the emphasized speech is likely to be at each time using a parameter for speech recognition, and the phoneme state posterior probability and a phoneme state input from the outside A parameter estimation step of estimating the parameter for mask estimation so that a predetermined distance criterion between the reference label and the reference label is minimized.

  According to the present invention, when performing speech recognition while adapting to the environment, the accuracy of speech recognition can be improved with limited learning data.

FIG. 1 is a diagram illustrating an example of the configuration of a mask estimation parameter estimation apparatus according to the first embodiment. FIG. 2 is a diagram illustrating an example of a configuration of a parameter estimation unit of the mask estimation parameter estimation apparatus according to the first embodiment. FIG. 3 is a flowchart showing a process flow of the mask estimation parameter estimation apparatus according to the first embodiment. FIG. 4 is a flowchart showing a process flow of the parameter estimation unit of the mask estimation parameter estimation apparatus according to the first embodiment. FIG. 5 is a diagram illustrating an example of a computer that realizes a mask estimation parameter estimation apparatus by executing a program.

  Hereinafter, embodiments of a mask estimation parameter estimation device, a mask estimation parameter estimation method, and a mask estimation parameter estimation program according to the present application will be described in detail with reference to the drawings. In addition, this invention is not limited by this embodiment.

[Configuration of First Embodiment]
First, the configuration of the mask estimation parameter estimation apparatus according to the first embodiment will be described with reference to FIG. FIG. 1 is a diagram illustrating an example of the configuration of a mask estimation parameter estimation apparatus according to the first embodiment. As illustrated in FIG. 1, the mask estimation parameter estimation apparatus 10 includes a time-frequency analysis unit 11, a mask estimation unit 12, a speech enhancement unit 13, a speech recognition unit 14, and a parameter estimation unit 15.

  In the first embodiment, the parameter estimation unit 15 interprets the mask estimation unit 12, the speech enhancement unit 13, and the speech recognition unit 14 as one calculation network. That is, the parameter estimation unit 15 estimates a parameter for mask estimation based on the output of the calculation network including the mask estimation unit 12, the speech enhancement unit 13, and the speech recognition unit 14. Hereinafter, each processing unit will be described.

  The time-frequency analysis unit 11 has a situation in which N acoustic signals including one first acoustic signal corresponding to the target sound source and N−1 second acoustic signals corresponding to noise are mixed. In FIG. 5, a short-time signal analysis is applied to each of M observation signals recorded at different positions to extract an observation signal for each time frequency point, and an M-dimensional vertical vector of the observation signal for each time frequency point. Construct an observation vector. The noise includes interference sound and background noise.

Assume that an observation feature vector obtained by using short-time signal analysis such as short-time Fourier transform is represented as y _{f, t} . However, t and f are time and frequency numbers, respectively, t is an integer from 1 to T, and f is an integer from 0 to F. At this time, it is assumed that the target sound source and the noise have sparsity, and there is at most one target sound source at each time frequency point, and the observation vector y _{f, t} at each time frequency point is _expressed by the following equation (1). Alternatively, it is known that modeling can be performed in any form of (2) (for example, see Non-Patent Document 1).

Here, Expression (1) represents a case where only the target sound source exists at the time frequency point, and Expression (2) represents a case where only the nth noise exists. s (f, t) is a time frequency component of the first acoustic signal, that is, the acoustic signal corresponding to the target sound source. Further, v _n (f, t) is a time-frequency component of the nth second acoustic signal among N−1, that is, the acoustic signal corresponding to the nth noise.

Based on the observation vector and the mask estimation parameter, the mask estimation unit 12 determines how much each of the N acoustic signals is included in the observation vector yf _{, t} for each time frequency point. Is estimated.

Specifically, the mask estimation unit 12 models the probability distribution of the observation vectors y _{f and t} for each frequency with a mixed distribution composed of N element distributions corresponding to each of the N acoustic signals. Is estimated as a mask corresponding to each of the N acoustic signals.

First, the mask estimation unit 12 represents the probability distribution of the observation vectors y _{f, t} at each time frequency point by Expression (3).

Here, Θ ^(s) is an element distribution parameter corresponding to the target sound source. Further, Θ ^(vn) is an element distribution parameter corresponding to the n-th noise. Α _f ^(s) is a weight parameter of the element distribution corresponding to the target sound source at the frequency f. Α _f ^(vn) is a weight parameter of an element distribution corresponding to the nth noise at the frequency f. Further, α _f ^(s) and α _f ^(vn) satisfy Expression (4).

At this time, the mask estimation unit 12 calculates a mask λ _{f, t} ^(s) corresponding to the target sound source at the time frequency point (f, t ^{) according} to Expression (5).

Equation (5) is obtained by using the observation vectors y _{f and t} as input and internal parameters Θ ^(s) , Θ ^(v1) ,..., Θ ^(vN−1) , α _f ^(s) _, α _f ^{( Since v1)} ,..., α _f ^(vN−1) can be interpreted as a calculation network for estimating the mask λ _{f, t} ^(s) , the calculation by the calculation network is expressed by Expression (6). However, ν is s or _{v n.}

The speech enhancement unit 13 obtains enhanced speech by multiplying the observation vector y _{f, t} and the mask for the first acoustic signal at each time frequency point. For example, as shown in Expression (7), the speech enhancement unit 13 uses the observation vector y _{f, t} ^{(m ′)} corresponding to the m′-th observation signal among the M observation signals and the mask λ _{f, t.} ^The emphasized speech is obtained by multiplying ^(s) at the time frequency point.

  Here, the calculation of Expression (7) is expressed as Expression (8).

The speech recognition unit 14 estimates a phoneme state posterior probability that indicates which phoneme state the emphasized speech is likely to be at each time, using speech recognition parameters learned in advance using learning data. Here, the calculation of the phoneme state posterior probability by the speech recognition unit 14 is expressed as in Expression (9). ^ I _t is a vector in which phoneme state posterior probabilities corresponding to I phoneme states at time t are arranged. In the following description, ^ a represents a symbol with a appended on a.

  From the equations (6), (8), and (9), the process of estimating the phoneme state posterior probability using the observation vector can be described as one computation network. The parameter estimation unit 15 retains the parameters and structure of the calculation network, so that a predetermined distance criterion between the phoneme state posterior probability and the reference label of the phoneme state input from the outside is minimized. Estimate parameters.

  Thereby, the mask estimation parameter estimation apparatus 10 of the first embodiment can optimize the mask estimation parameters according to the environment. In addition, the parameters optimized by the mask estimation parameter estimation apparatus 10 are parameters that were originally estimated from a small amount of data based on the likelihood maximization criterion. Therefore, the number of parameters is relatively small, and parameter estimation is performed using a small amount of data. Even in the case of performing parameter estimation, parameter estimation can be performed while preventing overlearning.

[Example]
The process of the mask estimation parameter estimation apparatus 10 will be described based on an embodiment. In the embodiment, it is assumed that an acoustic signal output from one target sound source is recorded with M microphones under noise. At this time, if the observation signal recorded by the microphone m is y ^(m) (τ), y ^(m) (τ) is an acoustic signal s ^{(m (m)} corresponding to the target sound source, as shown in Equation (10). ⁾ It is expressed by the sum of (τ) and the acoustic signal v ^(m) (τ) corresponding to noise.

The time-frequency analysis unit 11 receives all the observation signals recorded by the microphones, applies the short-time signal analysis for each observation signal y ^(m) (τ), and the signal feature Y ^(m) for each time frequency. Find (f, t). The time-frequency analysis unit 11 can use a technique such as a short-time discrete Fourier transform or a short-time discrete cosine transform as a technique for short-time signal analysis. The time frequency analysis unit 11 further obtains an observation vector y _{f, t} , which is a vector obtained by collecting the signals Y ^(m) (f, t) obtained at the respective time frequencies with respect to all microphones, using the expression (11). Configure as follows.

In the embodiment, in order to simplify the ^description without losing generality, the weight parameter of the element distribution corresponding to the target sound source is set to α ^(s) = α ^(v1) =... = Α ^(vN−1) = 1 / Suppose N. The mask estimation unit 12 models the observation vector at each time frequency point by a mixed distribution of two normal distributions for the target sound source and noise, respectively. At this time, when the distribution parameter φ _{f, t} ^(ν) and the mask estimation parameter R _f ^(ν) are given, the mask estimation unit 12 sets the posterior probability corresponding to each normal distribution to the equation (12). .

Here, using the update rule of the parameter φ _{f, t} ^(ν) described in Non-Patent Document 1, Expression (12) can be expressed as Expression (13).

From Equation (5) and Equation (13), the mask estimation unit 12 calculates the mask λ _{f, t} ^(s) for the mask target sound source as shown in Equation (14).

However, _{pf, t} ^(s) and _{pf, t} ^(v) are as Formula (15) and (16), respectively.

Since the calculation by the mask estimation unit 12 can be interpreted as a network for calculating a mask corresponding to the target sound source from the observation vector using R _f ^(ν) as an internal parameter, the calculation by the calculation network is expressed by Expression (17). . Equation (17) corresponds to Equation (6).

Also, as an example, the mask estimation unit 12 uses the probability distribution of the observation vectors y _{f, t} as N M-dimensional complex Gaussian distributions with an average of 0, and the covariance matrix has different values at each time. Modeling is performed with a mixed distribution consisting of an M-dimensional complex Gaussian distribution represented by the product of Hermitian matrices having scalar parameters and time-invariant parameters as elements. For example, φ _{f, t} ^(ν ) in equations (10) and (11) can be a scalar parameter that takes different values at each time, and R _f ^(ν) can be a time-invariant parameter.

The speech enhancement unit 13 receives the mask and uses the equation (18) to ^apply the mask λ _{f, t} ^(s) to the component y _{f, t} ^{(m ′)} recorded by the m′-th microphone that is the reference microphone. Is multiplied to calculate the emphasized speech ^ s _{f, t} . The speech enhancement unit 13 may calculate the enhanced speech ^ s _{f, t} by multiplying y _{f, t} ^{(m ′)} by a value obtained by multiplying the mask λ _{f, t} ^(s) to the β power.

The speech recognition unit 14 uses the parameters for speech recognition learned in advance based on the vector ^ s _t = [^ s _{1, t} , ..., ^ s _{F, t} ] in which the emphasized speech at each frequency is arranged. Then, the linear operation and the non-linear operation are repeated a plurality of times, and the phoneme state posterior probability ^ I = [^ I1 _{, t} ,..., ^ _{IK, t} ] is calculated for each time. The calculation by the voice recognition unit 14 at this time is expressed as in Expression (19).

  By the equations (17) to (19), the mask estimation unit 12, the speech enhancement unit 13, and the speech recognition unit 14 can be interpreted as one calculation network having the observation vector as an input and the phoneme state posterior probability as an output. . Here, estimation of parameters for mask estimation by the parameter estimation unit 15 will be described with reference to FIG. FIG. 2 is a diagram illustrating an example of a configuration of a parameter estimation unit of the mask estimation parameter estimation apparatus according to the first embodiment. As shown in FIG. 2, the parameter estimation unit 15 includes a mask estimation parameter initialization unit 151, a gradient calculation unit 152, a parameter holding unit 153, a parameter update unit 154, and a convergence determination unit 155.

In the embodiment, the recognition result when speech recognition is performed without performing speech enhancement is used as a binary reference label. At this time, in the parameter estimator 15, the objective function for parameter update is a cross between the phoneme state posterior probabilities ^ I _t and the reference labels I _t = [I _{1, t} ,..., I _{K, t} ]. Entropy can be defined as shown in Equation (20).

In addition, the cross entropy of Formula (20) is an example of the distance reference | standard which the parameter estimation part 15 minimizes. The mask estimation parameter initialization unit 151 determines the initial value of the mask estimation parameter R _f ^(ν) and the learning rate γ. The mask estimation parameter R _f ^(ν) includes R _f ^(s) and R _f ^(s) . Note that the mask estimation parameter initialization unit 151 may use the initial value of the mask estimation parameter R _f ^(ν) as a unit matrix, or may obtain it using a likelihood maximization criterion described in Non-Patent Document 1. . The parameter holding unit 153 holds the parameters of the voice enhancement unit 13 and the parameters of the voice recognition unit 14.

The parameter updating unit 154 updates the mask estimation parameter R _f ^(ν) using Expression (21) based on the principle of the steepest descent method. In this case, what is actually updated is the inverse matrix of the parameter R _f ^(ν) for mask estimation.

Here, as described above, since the mask estimation unit 12, the speech enhancement unit 13, and the speech recognition unit 14 can be interpreted as one calculation network, the gradient calculation unit 152 includes the phoneme state posterior probability ^ I _t , the reference label I _t and the parameter held by the parameter holding unit 153 are received, and the gradient ∂L (I _t , ^ I _t ) / ∂ {R _f ^{(ν) −1} } ^* in Equation (21) is Is calculated as shown in Equation (22).

  The convergence determination unit 155 determines whether or not the objective function has converged as a result of the update by the parameter update unit 154. When the convergence determination unit 155 determines that the convergence has been completed, the parameter estimation unit 15 outputs the estimated mask estimation parameter. When the convergence determination unit 155 determines that the convergence has not occurred, the parameter estimation unit 15 further repeats the processing by the gradient calculation unit 152 and the parameter update unit 154 using the updated mask estimation parameter. The convergence determination unit 155 may determine that the process has converged when the process is repeated a predetermined number of times.

Accordingly, the parameter estimation unit 15 can obtain the phoneme state posterior probability ^ I _t, a mask estimation parameters such as locally minimize cross entropy between the reference label I _t.

When the speech recognition unit 14 is configured by a neural network, the gradient calculation unit 152 calculates the gradient ∂L (I _t , ^ I _t ) / ∂ ^ s _{f, t} in the equation (22) as the neural network. Can be calculated using a method based on backpropagation that is used in estimating the parameters. For example, the gradient calculation unit 152 calculates the gradient ∂ ^ s _{f, t} / ∂λ _{f, t} ^(x) as the equation (23) based on the equation (18).

For example, the gradient calculation unit 152 calculates the gradient ∂λ _{f, t} ^(x) / ^） p _{f, t} ^(ν) as the equation (24) or (25) based on the equation (14).

For example, the gradient calculation unit 152 calculates the gradient ∂p _{f, t} ^(ν) / ^） {R _f ^{(ν) −1} } ^* as the equation (26) based on the equations (15) and (16). To do.

[Process of First Embodiment]
A processing flow of the mask estimation parameter estimation apparatus 10 will be described with reference to FIG. FIG. 3 is a flowchart showing a process flow of the mask estimation parameter estimation apparatus according to the first embodiment.

  As shown in FIG. 3, first, the time-frequency analysis unit 11 of the mask estimation parameter estimation apparatus 10 performs time-frequency analysis on the acoustic signal corresponding to the target sound source and noise, and obtains an observation vector (step S11). . Next, the mask estimation unit 12 estimates a mask for speech enhancement based on the mask estimation parameter and the observation vector (step S12).

  The speech enhancement unit 13 multiplies the observation vector and the mask estimated by the mask estimation unit 12 to obtain enhanced speech (step S13). Then, the voice recognition unit 14 performs voice recognition using the emphasized voice and the parameters for voice recognition (step S14). Then, the parameter estimation unit 15 uses the mask estimation unit 12, the speech enhancement unit 13, and the speech recognition unit 14 as one calculation network, and estimates the parameters for mask estimation so that the speech recognition result is close to the reference label (step) S15).

  Next, the process of the parameter estimation part 15 is demonstrated using FIG. FIG. 4 is a flowchart showing a process flow of the parameter estimation unit of the mask estimation parameter estimation apparatus according to the first embodiment. As shown in FIG. 4, the mask estimation parameter initialization unit 151 of the parameter estimation unit 15 determines an initial value of the mask estimation parameter (step S151). Next, the gradient calculation unit 152 receives the speech state posterior probability, the reference label, and the parameters of the speech enhancement unit 13 and the speech recognition unit 14, and calculates the gradient of the distance standard between the speech state posterior probability and the reference label. Calculation is performed (step S152). The parameter updating unit 154 updates the parameters for mask estimation so that the distance reference becomes small (step S153).

  When the convergence determination unit 155 determines that the mask estimation parameters have converged (step S154, Yes), the parameter estimation unit 15 ends the process. When the convergence determination unit 155 determines that the parameter for mask estimation has not converged (No in step S154), the parameter estimation unit 15 returns the process to step S152 and uses the updated mask estimation parameter, The processing by the gradient calculation unit 152 and the parameter update unit 154 is further repeated.

[Effect of the first embodiment]
The time-frequency analysis unit 11 has a situation in which N acoustic signals including one first acoustic signal corresponding to the target sound source and N−1 second acoustic signals corresponding to noise are mixed. In FIG. 5, a short-time signal analysis is applied to each of M observation signals recorded at different positions to extract an observation signal for each time frequency point, and an M-dimensional vertical vector of the observation signal for each time frequency point. Construct an observation vector. Further, the mask estimation unit 12 indicates how much each of the N acoustic signals is included in the observation vector for each time frequency point based on the observation vector and the mask estimation parameter. Estimate the mask. Further, the speech enhancement unit 13 acquires the enhanced speech by multiplying the observation vector and the mask for the first acoustic signal at each time frequency point. In addition, the speech recognition unit 14 estimates a phoneme state posterior probability that indicates which phoneme state the emphasized speech is likely to be at each time, using speech recognition parameters learned in advance using the learning data. Further, the parameter estimation unit 15 estimates a parameter for mask estimation so that a predetermined distance criterion between the phoneme state posterior probability and the reference label of the phoneme state input from the outside is minimized.

  Thus, in the first embodiment, it is not necessary to reflect the result of speech recognition by the speech recognition unit 14 in the estimation of parameters for mask estimation, and it is not necessary to update the parameter of the speech recognition unit 14. When performing speech recognition while adapting, the accuracy of speech recognition can be improved with limited learning data.

  The mask estimation unit 12 models the probability distribution of the observation vector for each frequency by a mixed distribution including N element distributions corresponding to each of the N acoustic signals, and sets the posterior probability of the element distribution to N pieces. You may estimate as a mask corresponding to each of an acoustic signal. This makes it possible to estimate a mask for each of the target sound source and the acoustic signal corresponding to the noise.

  The mask estimator 12 determines the probability distribution of the observation vector as N M-dimensional complex Gaussian distributions having an average of 0, and a covariance matrix having different values at each time and a time-invariant parameter. May be modeled by a mixed distribution consisting of an M-dimensional complex Gaussian distribution represented by a product of Hermitian matrices having elements as.

  In general, an acoustic signal corresponding to a target sound source mainly comes from a sound source direction as viewed from a microphone, and noise comes from all directions. The Hermite matrix has the property that the subspace corresponding to the sound source direction has the largest eigenvalue and the other subspaces have relatively small eigenvalues. Thus, it becomes clear which acoustic signal the estimated mask corresponds to.

Here, a confirmation experiment using the conventional method and the first embodiment performed to confirm the effect of the present invention will be described. In the confirmation experiment, the learning rate γ was 10 ⁵ , the initial value of R _f ^(ν) was a value obtained by the likelihood maximization criterion described in Non-Patent Document 1, and the number of update rule iterations was 30. Speech enhancement was performed by multiplying the masks as they were.

In the confirmation experiment, in a situation where there is background noise in a bus, cafe, etc., one speaker is reading a sentence into the tablet, and M = 6 microphones attached to the tablet. Voice recognition was performed on the recorded signals. The word error rate when speech recognition is performed using the conventional method and when speech recognition is performed using the first embodiment is shown below.
(1) When speech recognition is performed without performing speech enhancement: 24.66 (%)
(2) After estimating the distribution parameters using the likelihood maximization criterion described in Non-Patent Document 1 and then performing speech recognition after performing speech enhancement by masking: 19.88 (%)
(3) When speech recognition is performed after re-estimating some of the parameters of the speech recognition unit by the method described in Non-Patent Document 2: 24.10 (%)
(4) When the speech recognition is performed after estimating the distribution parameter by the method of the first embodiment and performing speech enhancement by masking: 18.35 (%)
As a result of the confirmation experiment, the word error rate was the smallest in the case of (4). Thus, according to the first embodiment, it can be said that the voice recognition accuracy can be improved as compared with the conventional method.

[System configuration, etc.]
Further, each component of each illustrated apparatus is functionally conceptual, and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured. Furthermore, all or a part of each processing function performed in each device may be realized by a CPU and a program that is analyzed and executed by the CPU, or may be realized as hardware by wired logic.

  Also, among the processes described in this embodiment, all or part of the processes described as being performed automatically can be performed manually, or the processes described as being performed manually can be performed. All or a part can be automatically performed by a known method. In addition, the processing procedure, control procedure, specific name, and information including various data and parameters shown in the above-described document and drawings can be arbitrarily changed unless otherwise specified.

[program]
As one embodiment, the mask estimation parameter estimation apparatus 10 can be implemented by installing a mask estimation parameter estimation program for executing the above mask estimation parameter estimation as package software or online software on a desired computer. For example, the information processing apparatus can function as the mask estimation parameter estimation apparatus 10 by causing the information processing apparatus to execute the mask estimation parameter estimation program. The information processing apparatus referred to here includes a desktop or notebook personal computer. In addition, the information processing apparatus includes mobile communication terminals such as smartphones, mobile phones and PHS (Personal Handyphone System), and slate terminals such as PDA (Personal Digital Assistant).

  The mask estimation parameter estimation apparatus 10 can also be implemented as a mask estimation parameter estimation server apparatus that uses a terminal device used by a user as a client and provides the client with services related to the above mask estimation parameter estimation. For example, a mask estimation parameter estimation server device is implemented as a server device that provides a mask estimation parameter estimation service that receives each parameter of a calculation network, a speech recognition result, and a reference label and outputs a mask estimation parameter. . In this case, the mask estimation parameter estimation server apparatus may be implemented as a Web server, or may be implemented as a cloud that provides services related to the above mask estimation parameter estimation through outsourcing.

  FIG. 5 is a diagram illustrating an example of a computer that realizes a mask estimation parameter estimation apparatus by executing a program. The computer 1000 includes a memory 1010 and a CPU 1020, for example. The computer 1000 also includes a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These units are connected by a bus 1080.

  The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM 1012. The ROM 1011 stores a boot program such as BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1090. The disk drive interface 1040 is connected to the disk drive 1100. For example, a removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to a mouse 1110 and a keyboard 1120, for example. The video adapter 1060 is connected to the display 1130, for example.

  The hard disk drive 1090 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. That is, a program that defines each process of the mask estimation parameter estimation apparatus 10 is implemented as a program module 1093 in which a code executable by a computer is described. The program module 1093 is stored in the hard disk drive 1090, for example. For example, a program module 1093 for executing processing similar to the functional configuration in the mask estimation parameter estimation apparatus 10 is stored in the hard disk drive 1090. Note that the hard disk drive 1090 may be replaced by an SSD.

  The setting data used in the processing of the above-described embodiment is stored as program data 1094 in, for example, the memory 1010 or the hard disk drive 1090. Then, the CPU 1020 reads the program module 1093 and the program data 1094 stored in the memory 1010 and the hard disk drive 1090 to the RAM 1012 and executes them as necessary.

  The program module 1093 and the program data 1094 are not limited to being stored in the hard disk drive 1090, but may be stored in, for example, a removable storage medium and read out by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and the program data 1094 may be stored in another computer connected via a network (LAN, WAN (Wide Area Network), etc.). The program module 1093 and the program data 1094 may be read by the CPU 1020 from another computer via the network interface 1070.

DESCRIPTION OF SYMBOLS 10 Mask estimation parameter estimation apparatus 11 Time frequency analysis part 12 Mask estimation part 13 Speech enhancement part 14 Speech recognition part 15 Parameter estimation part 151 Mask estimation parameter initialization part 152 Gradient calculation part 153 Parameter holding part 154 Parameter update part 155 Convergence Judgment part

Claims (7)

N acoustic signals including one first acoustic signal corresponding to the target sound source and N−1 second acoustic signals corresponding to noise (where N is an integer of 2 or more) In a mixed situation, a short-time signal analysis is applied to each of M observation signals (where M is an integer of 2 or more) recorded at different positions to extract observation signals at each time frequency point, A time-frequency analyzer that constitutes an observation vector that is an M-dimensional vertical vector of observation signals for each time-frequency point;
Based on the observation vector and a parameter for mask estimation, a mask representing how much each of the N acoustic signals is included in the observation vector for each time frequency point is estimated. A mask estimation unit;
A speech enhancement unit that obtains enhanced speech by multiplying the observation vector and the mask for the first acoustic signal at each of the time frequency points;
Using a speech recognition parameter learned in advance using learning data, a speech recognition unit that estimates a phoneme state posterior probability indicating which phonetic state the emphasized speech is likely to be at each time;
A parameter estimation unit that estimates the parameter for mask estimation so that a predetermined distance criterion between the phoneme state posterior probability and a reference label of the phoneme state input from the outside is minimized;
The parameter estimation apparatus for mask estimation characterized by having.   The mask estimation unit models the probability distribution of the observation vector for each frequency by a mixed distribution composed of N element distributions corresponding to each of the N acoustic signals, and sets the posterior probability of the element distribution as The mask estimation parameter estimation apparatus according to claim 1, wherein the estimation is performed as a mask corresponding to each of the N acoustic signals.   The mask estimation unit is a scalar parameter and a time-invariant parameter in which the probability distribution of the observation vector is N M-dimensional complex Gaussian distributions having an average of 0, and the covariance matrix takes different values at each time. 3. The parameter estimation apparatus for mask estimation according to claim 2, wherein modeling is performed with a mixed distribution composed of an M-dimensional complex Gaussian distribution represented by a product of Hermitian matrices having elements as elements. A parameter estimation method for mask estimation executed by a parameter estimation apparatus for mask estimation,
N acoustic signals including one first acoustic signal corresponding to the target sound source and N−1 second acoustic signals corresponding to noise (where N is an integer of 2 or more) In a mixed situation, a short-time signal analysis is applied to each of M observation signals (where M is an integer of 2 or more) recorded at different positions to extract observation signals at each time frequency point, A time-frequency analysis step of constructing an observation vector that is an M-dimensional vertical vector of the observation signal for each time-frequency point;
Based on the observation vector and a parameter for mask estimation, a mask representing how much each of the N acoustic signals is included in the observation vector for each time frequency point is estimated. A mask estimation step;
A speech enhancement step of acquiring enhanced speech by multiplying the observation vector and the mask for the first acoustic signal at each of the time frequency points;
Using a speech recognition parameter learned in advance using learning data, a speech recognition step of estimating a phoneme state posterior probability indicating which phonetic state the emphasized speech is likely to be at each time;
A parameter estimation step of estimating the parameter for mask estimation so that a predetermined distance criterion between the phoneme state posterior probability and a reference label of the phoneme state input from the outside is minimized;
The parameter estimation method for mask estimation characterized by including.   The mask estimation step models the probability distribution of the observation vector for each frequency by a mixed distribution composed of N element distributions corresponding to each of the N acoustic signals, and sets the posterior probability of the element distribution as The mask estimation parameter estimation method according to claim 4, wherein the estimation is performed as a mask corresponding to each of the N acoustic signals.   In the mask estimation step, the probability distribution of the observation vector is N M-dimensional complex Gaussian distributions whose mean is 0, and the covariance matrix is a scalar parameter and a time-invariant parameter that take different values at each time. 6. The parameter estimation method for mask estimation according to claim 5, wherein modeling is performed with a mixed distribution composed of an M-dimensional complex Gaussian distribution represented by a product of Hermitian matrices having elements as elements.   A mask estimation parameter estimation program for causing a computer to function as the mask estimation parameter estimation apparatus according to any one of claims 1 to 3.

Images