JP6734233B2 - Signal processing device, case model generation device, collation device, signal processing method, and signal processing program - Google Patents

Description

The present invention relates to a signal processing device, a case model generation device, a matching device, a signal processing method, and a signal processing program.

Conventionally, in a voice recognition system, a hearing aid, a TV conference system, a machine control interface, a music information processing system for music search and transcription, etc., a technique for picking up an acoustic signal and extracting a component of a target audio signal is used. Has been done.

Generally, when an acoustic signal is picked up in a real environment with noise or reverberation, not only a voice signal for the purpose of picking up sound but also a signal on which noise and reverberation (acoustic distortion) are superimposed is observed. However, when these noises and reverberations are superimposed on the signal, it becomes difficult to extract the original component of the audio signal, which causes a significant reduction in the clarity and intelligibility of the audio signal. As a result, it becomes difficult to extract the nature of the original voice signal, and, for example, the recognition rate of the voice recognition system decreases. In order to prevent this reduction in recognition rate, it is necessary to devise a method for removing the superimposed acoustic distortion.

Therefore, conventionally, by using a case model represented by a Gaussian Mixture Model (GMM), the similarity with the feature amount obtained by converting the input speech is examined, and the case model showing a high degree of similarity is used as a speech signal. A candidate signal processing device has been proposed (for example, see Non-Patent Document 1).

A conventional signal processing device performs signal processing using a case model represented by a mixture distribution model learned in advance. The case model is, for example, the amplitude spectrum of clean speech corresponding to each case, and a sequence of Gaussian mixture distribution indices (segments) that gives the maximum likelihood to the feature amount (eg, Mel frequency cepstrum coefficient) for each frame. including. The signal processing device uses the segment of the feature amount generated from the input voice to obtain the amplitude spectrum of the clean voice most similar to the input voice, and from the case models obtained in advance, the segment that gives the maximum posterior probability. To explore.

It has been reported that such a conventional signal processing device can remove noise, which has been difficult until then, and which has a very large time change. The noise that changes significantly over time is noise such as an alarm sound of an alarm clock with respect to background noise.

J. Ming and R. Srinivasan, and D. Crookes, “A Corpus-Based Approach to Speech Enhancement From Nonstationary Noise”, IEEE Transactions on Audio, Speech, and Language Processing, Vol.19, No.4, pp.822- 836, May 2011 G. Hinton, L. Deng, D. Yu, GE Dahl, A.-r. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, TN Sainath, and B. Kingsbury, “Deep Neural Networks for Acoustic Modeling in Speech Recognition: The shared views of four research groups”, IEEE Signal Processing Magazine, pp. 82-97, Nov. 2012.

However, the mel frequency cepstrum coefficient used for segment search is a simple feature amount obtained from the amplitude spectrum. Therefore, when the input signal includes noise and reverberation, the mel frequency cepstrum coefficient also includes the influence of noise and reverberation, and the segment search in the conventional signal processing device is not always highly accurate.

Further, although the case models are prepared by assuming various acoustic distortion environments, it is difficult to prepare case models corresponding to all the acoustic distortion environments in reality. For this reason, in the conventional signal processing device, there is a case where a segment having a high degree of similarity to the segment of the feature amount generated from the input signal cannot be searched from the case model.

Therefore, in the conventional signal processing device, since the feature amount used for the search is affected by noise and reverberation, there is a limit to the accuracy of searching the feature amount of clean speech similar to the input signal.

The present invention has been made in view of the above, and provides a signal processing device, a case model generation device, a collation device, a signal processing method, and a signal processing program for accurately searching for clean speech similar to an input signal. With the goal.

In order to solve the above-mentioned problems and achieve the object, the signal processing device according to the present invention inputs voice or clean voice including noise or acoustic distortion and outputs it using an acoustic model based on Neural Network (NN). The storage unit that stores the generated case model, the feature amount generation unit that generates the feature amount from the input signal, the output result that is output using the acoustic model based on the NN with the feature amount as an input, and the storage unit that stores the output result. And outputs a emphasized speech obtained by multiplying the input signal by a matching unit that obtains a clean speech feature amount corresponding to the input signal and a filter including the clean speech feature amount obtained by the matching unit. And an output unit.

In addition, in order to solve the above-mentioned problems and achieve the object, the case model generation device according to the present invention uses a feature amount generation unit that generates a feature amount from a learning input signal, and a DNN (feature amount). HMM (Hidden Markov Model) based on Deep Neural Network) A learning unit for learning an acoustic model, and the likelihood of the HMM state output by the HMM acoustic model based on DNN, based on the feature amount for each time frame. And a maximum likelihood HMM state calculation unit that calculates, as a case model, a series of HMM state indexes that give the maximum likelihood.

Further, in order to solve the above-mentioned problems and achieve the object, the matching device according to the present invention inputs the feature amount of the input signal into the HMM acoustic model based on DNN, and outputs the output result by the HMM acoustic model based on DNN. , A speech including noise or acoustic distortion or a clean speech is collated with a case model learned by using an HMM acoustic model based on DNN, and a collation unit for obtaining a clean speech feature amount corresponding to an input signal is provided. ..

According to the present invention, it is possible to accurately search for clean speech that is similar to an input signal.

FIG. 1 is a diagram illustrating an example of a functional configuration of a signal processing device according to an embodiment. FIG. 2 is a diagram for explaining an example of the segment. FIG. 3 is a diagram for explaining the processing of the matching unit shown in FIG. FIG. 4 is a flowchart showing a processing procedure of a signal processing method executed by the signal processing device shown in FIG. FIG. 5 is a block diagram showing an example of the functional configuration of the case model generation device according to the embodiment. FIG. 6 is a flowchart showing a processing procedure of a case model generation process by the case model generation device shown in FIG. FIG. 7 is a diagram illustrating an example of a computer in which a signal processing device or a case model generation device is realized by executing a program.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. The present invention is not limited to this embodiment. In the description of the drawings, the same parts are designated by the same reference numerals.

[Embodiment]
First, the signal processing device according to the embodiment will be described. This signal processing device is a device that removes acoustic distortion from an input signal including noise and reverberation (acoustic distortion) and outputs a clear emphasized speech signal.

[Configuration of signal processing device]
FIG. 1 is a diagram illustrating an example of a functional configuration of a signal processing device according to an embodiment. In the signal processing device 100 according to the first embodiment, for example, a predetermined program is read into a computer including a ROM (Read Only Memory), a RAM (Random Access Memory), a CPU (Central Processing Unit), etc. It is realized by executing a predetermined program.

As shown in FIG. 1, the signal processing device 100 includes a case model storage unit 101 (storage unit), a Fourier transform unit 102, a feature amount generation unit 103, a matching unit 104 (matching unit), and a speech enhancement filtering unit 105 (output unit. ) Has. The signal processing apparatus 100 performs a sequence (segment) search using a case model represented by an HMM (Hidden Markov Model) acoustic model (hereinafter referred to as a DNN-HMM acoustic model) based on DNN (Deep Neural Network). The DNN-HMM acoustic model has high noise immunity.

The case model storage unit 101 receives the voice or clean voice including noise or acoustic distortion as an input, and stores the case model Ms output using the DNN-HMM acoustic model. Specifically, the case model storage unit 101 stores the clean voice data corresponding to the case and the case model Ms. The clean voice data is, for example, an amplitude spectrum of clean voice corresponding to a case. Further, the case model is represented by a sequence of s (maximum likelihood HMM state sequence) that is an index of the HMM state that gives the maximum likelihood to the feature amount for each time frame. The case model Ms also includes the prior probability P(s) of the HMM state s. This is because the a priori probability P(s) of the HMM state s is used in the matching process in the matching unit 104, as will be described later.

Here, the case model Ms is generated in advance by the case model generation device 200 (described later) and stored in the case model storage unit 101. The case model generation apparatus 200 uses a large amount of clean speech obtained from a speech corpus and the like and noise and reverberation data (noise signal waveform, room impulse response, etc.) obtained in various environments, in various environments. Of the observed signal of FIG. 3 is simulated as a speech signal for learning, and the simulated observed signal converted into the feature amount region is learned by the DNN-HMM acoustic model (for details, refer to Non-Patent Document 2). To generate.

The DNN-HMM acoustic model has a function of estimating the HMM state number corresponding to the input speech feature amount. One state number of the HMM corresponds to the head portion, the middle portion, or the trailing portion of one phoneme such as “a”, “i”, and “u”, and is usually defined by a number of 3000 to 10000. It is often done. This DNN-HMM acoustic model outputs the likelihood of the HMM state s of about 3000 to 10000 at the output layer after converting the input feature amount of speech into a non-linear feature amount at the intermediate layer of a plurality of nodes.

Therefore, in the case model generation device 200 (described later), the maximum likelihood for each time frame i is calculated from the feature amount of the learning voice signal based on the DNN-HMM acoustic model (hereinafter, referred to as g _s ). A sequence of HMM state indexes (maximum likelihood HMM state sequence) s _i that gives a degree is obtained. The time series (segment) of the obtained index s _i becomes one of the case models Ms. The case model Ms is expressed as shown in the following equation (1) using the set of maximum likelihood HMM state series s _i and the DNN-HMM acoustic model g _s .

Incidentally, it s _i is the index of the HMM state to provide maximum likelihood for the feature amount k _i of i-th frame. I represents the total number of frames of the audio signal for learning. For example, assuming 1 hour of learning data, I=3.5×10 ⁵ .

Then, an example of the segments included in the case model will be described. FIG. 2 is a diagram for explaining an example of the segment. For example, each cell of the segment shown in FIG. 2 corresponds to the i-th time frame of the I frame. The number in each cell represents the index s _{i of the} HMM state that gives the maximum likelihood.

Next, returning to FIG. 1, the Fourier transform unit 102 will be described. The Fourier transform unit 102 transforms the input signal into an amplitude spectrum for each frame. An audio signal including noise and reverberation is input to the Fourier transform unit 102 as an input signal. First, the Fourier transform unit 102 cuts out waveform data of an input signal with a short time width. For example, the Fourier transform unit 102 multiplies a window function such as a short-time Hamming window of about 30 (msec) to cut out the input signal in a short time width. Then, the Fourier transform unit 102 executes a discrete Fourier transform process on the cut input signal to convert it into an amplitude spectrum. The amplitude spectrum is the amplitude data of the frequency spectrum. The Fourier transform unit 102 inputs the transformed amplitude spectrum to the feature amount generation unit 103 and the speech enhancement filtering unit 105.

The feature quantity generation unit 103 generates a feature quantity y _t from the amplitude spectrum output from the Fourier transform unit 102. In other words, the feature quantity generation unit 103 generates a segment of the feature quantity y _t from the amplitude spectrum input from the Fourier transform unit 102. Note that t is a frame to be processed. The feature amount generation unit 103 converts all the amplitude spectra output from the Fourier transform unit 102 into, for example, mel frequency cepstrum coefficients. As a result, the input signal is represented as a segment of the feature quantity vector for each frame.

Here, the commonly used mel frequency cepstrum coefficient is of the order of 10 to 20. In the signal processing device 100, in order to accurately represent the case model Ms, a mel frequency cepstrum coefficient having an order higher than a commonly used order (for example, about 30 to 100) is used. Therefore, the feature amount generation unit 103 converts all the amplitude spectra output from the Fourier transform unit 102 into, for example, mel frequency cepstrum coefficients of the order of 30 to 100. The feature amount generation unit 103 may use a feature amount other than the mel frequency cepstrum coefficient (for example, cepstrum coefficient or the like). The feature amount generation unit 103 inputs the generated feature amount y _t to the matching unit 104.

The matching unit 104 receives the feature amount y _t as an input, and collates (matches) the output result output using the DNN-HMM acoustic model g _s with the case model Ms stored in the case model storage unit 101. The clean speech feature amount corresponding to the input signal is obtained. The matching unit 104 collects a clean voice corresponding to the case model Ms showing a high degree of similarity with respect to the output result from the DNN-HMM acoustic model of the input feature value y _t of the input voice, and a voice signal candidate for the purpose of collecting the voice. To go.

Specifically, the matching unit 104 inputs the feature amount y _t of the input speech into the DNN-HMM acoustic model, and stores the likelihood of the HMM state s output by the DNN-HMM acoustic model and the case model storage unit 101. The maximum likelihood HMM state series s _{i of} the case model described above is collated, and the clean speech feature amount corresponding to the case model Ms showing a high degree of similarity is obtained as the feature amount of the clean speech corresponding to the input signal.

In other words, the matching unit 104 searches the case model in the case model storage unit 101 for a segment having the highest likelihood with respect to the likelihood of the HMM state output from the DNN-HMM acoustic model. The matching unit 104 inputs the information about the segment in the case model found by the search to the voice enhancement filtering unit 105. The details of the processing of the matching unit 104 will be described later.

The voice emphasis filtering unit 105 outputs an emphasis voice signal obtained by multiplying an input signal by a filter including the clean voice feature amount obtained by the matching unit 104. Specifically, the speech enhancement filtering unit 105 regards the amplitude spectrum of the clean speech corresponding to the feature amount of the segment of the case model Ms searched by the matching unit 104 as the amplitude spectrum of the clean speech most similar to the input signal, The amplitude spectrum of this clean voice is read from the case model storage unit 101. Subsequently, the voice enhancement filtering unit 105 creates a filter for voice enhancement using the read amplitude spectrum of the clean voice, and filters the input signal using the filter. As a result, the speech enhancement filtering unit 105 outputs an enhanced speech signal in which acoustic distortion is removed from the input signal.

[Processing of matching unit]
Next, the processing of the matching unit 104 will be described in detail. FIG. 3 is a diagram for explaining the processing of the matching unit 104 shown in FIG.

As shown in FIG. 3, the input to the matching unit 104 is the feature amount y _t of the input voice affected by noise and reverberation. y _t is, for example, a mel frequency cepstrum coefficient, a filter bank coefficient, or the like.

The mel frequency cepstrum coefficient used for speech recognition has about 13 dimensions, and its Δ and ΔΔ coefficients are often used at the same time. Therefore, the total dimension of the feature amount y _t is 39 times, which is three times as large as 13 dimensions. When input to the DNN-HMM acoustic model, not only the equal frames but also a total of 11 frame feature amounts including 5 frames before and after the same frame are often input at one time. In that case, the number of dimensions of the feature amount y _t is 429, which is 11 times as large as 39.

Further, when the input is a filter bank coefficient, the number of dimensions is further increased, and the basic number of dimensions is 40, and by taking Δ and ΔΔ into consideration, the number is 120, which is three times 49. Furthermore, by considering a total of 11 frames including 5 frames before and after the corresponding frame, a total of 1320-dimensional feature amount y _t is obtained.

The feature amount y _t is input to the DNN-HMM acoustic model in the matching unit 104. Incidentally, the number of nodes of the input layer of DNN-HMM acoustic model is equal to the number of dimensions of the feature _{y t.} Then, the input feature amount y _t is subjected to non-linear feature amount conversion by, for example, 2048 nodes, typically about 5 to 10 intermediate layers, and then, at the output layer, for example, an HMM of about 3000 to 10000 nodes. The likelihood p(y _t |s) of the state is obtained. Specifically, p(y _t |s) is calculated by the following equations (2) and (3).

Here, z _s ^L (y _t ) is the activity of the s-th node (corresponding to the HMM state s) of the output layer (L-th layer of the DNN-HMM acoustic model) when the feature amount y _t is given. It is a value. P(s) is the prior probability of the HMM state s. P(s) is included in the case model Ms. w _s,r ^L is a weighting coefficient from the r-th node of the final intermediate layer (the (L-1)-th layer of the DNN-HMM acoustic model) to the s-th node of the output layer (L-th layer). .. f(•) is an activation function (typically a sigmoid function). b _s ^L is a bias value of the sth node of the output layer (Lth layer). It should be noted that the method of obtaining p(y _t |s) from y _t by the DNN-HMM acoustic model is described in detail in Non-Patent Document 2, for example.

The matching unit 104 performs matching processing for matching the likelihood p(y _t |s) of the HMM state output from the output layer of the DNN-HMM acoustic model with the case model Ms stored in the case model storage unit 101. .. Then, the matching unit 104 searches for the likelihood p(y _t |s) of the HMM state output from the output layer and the case model Ms exhibiting a high degree of similarity, and cleans the case model Ms searched for. The voice is used as a voice signal candidate for collecting sound.

An example of a method of searching the maximum likelihood HMM state series of the case model Ms closest to the likelihood p(y _t |s) of the HMM state of the feature amount of the input signal will be described. For example, the matching unit 104 associates the likelihood of the HMM state output from the DNN-HMM acoustic model from the case models in the case model storage unit 101 with each of the segments that are the maximum likelihood HMM state series, and outputs the result. The segment having the highest likelihood is extracted with respect to the likelihood of the HMM state. In other words, the matching unit 104 applies the HMM state number sequence output from the DNN-HMM acoustic model from the case models in the case model storage unit 101 to each segment of the maximum likelihood HMM state series, and outputs the HMM state number sequence. The segment having the highest likelihood with respect to the HMM state number sequence is extracted.

The matching unit 104 also calculates the distance between the likelihood p(y _t |s) of the HMM state of the feature amount of the input signal and the maximum likelihood HMM state sequence that is a segment in the case model Ms, for example, Euclidean distance. The case model Ms may be searched based on

Then, the matching unit 104 outputs information about the searched segment of the case model Ms, that is, the segment of the case model Ms that is considered to give the clean speech sequence most similar to the clean speech included in the input signal, to the speech enhancement filtering unit. Input to 105. Thereby, the voice enhancement filtering unit 105 creates a filter for voice enhancement by using the amplitude spectrum of the clean voice in the case model storage unit 101 corresponding to the segment, and filters the input signal by the filter. , Outputs the emphasized voice signal.

As described above, in the signal processing device 100 according to the embodiment, the segment search is performed using the likelihood p(y _t |s) of the HMM state output from the DNN-HMM acoustic model. This DNN-HMM acoustic model has high noise resistance. In other words, the DNN-HMM acoustic model can estimate the HMM state number with high accuracy even if the input speech feature amount is affected by noise or reverberation. Therefore, in the signal processing device 100, the DNN-HMM acoustic model having high noise resistance is used to perform a segment search that is robust against noise and reverberation, that is, a segment search that is less susceptible to noise and reverberation. Therefore, next, the procedure of the signal processing method using the DNN-HMM acoustic model in the signal processing device 100 will be described.

[Signal Processing Method in Signal Processing Device]
Next, a signal processing method in the signal processing device 100 will be described. FIG. 4 is a flowchart showing a processing procedure of a signal processing method executed by the signal processing device 100 shown in FIG.

First, the Fourier transform unit 102 performs a Fourier transform process (step S1) of transforming an input signal into an amplitude spectrum. The feature amount generation unit 103 performs a feature amount generation process (step S2) of generating a feature amount such as a mel frequency cepstrum coefficient from the amplitude spectrum output from the Fourier transform unit 102.

The matching unit 104 inputs the feature amount generated by the feature amount generation unit 103 to the DNN-HMM acoustic model, the likelihood of the HMM state output from the DNN-HMM acoustic model, and the case model Ms of the case model storage unit 101. The maximum likelihood HMM state sequence is matched, and a matching process is performed in which a clean voice corresponding to the case model Ms showing a high degree of similarity is set as a voice signal candidate for sound pickup (step S3).

The voice enhancement filtering unit 105 creates a filter for voice enhancement using the amplitude spectrum of clean voice corresponding to the feature amount of the segment of the case model Ms searched by the matching unit 104, and multiplies the input signal by the filter. A voice emphasis filtering process (step S4) of outputting emphasized voice is performed.

[Configuration of case model creation device]
Next, the case model generation device 200 that generates the case model Ms stored in the case model storage unit 101 of the signal processing device 100 will be described. Also in this case model generation device 200, for example, learning is performed using a DNN-HMM acoustic model having high noise resistance with respect to the characteristic amount y _t such as the mel frequency cepstrum coefficient generated from the learning speech signal. , The case model Ms is generated.

FIG. 5 is a block diagram showing an example of the functional configuration of the case model generation device 200. The case model generation device 200 illustrated in FIG. 5 is realized by, for example, a computer including a ROM, a RAM, a CPU, or the like reading a predetermined program and the CPU executing the predetermined program. The case model generation device 200 includes a Fourier transform unit 201, a feature amount generation unit 202, a DNN-HMM acoustic model learning unit 203 (learning unit), and a maximum likelihood HMM state calculation unit 204.

First, a learning voice signal input to the case model generation device 200 will be described. The signal input to the case model generation device 200 is a voice signal of various noise/reverberation environments. The audio signal of the clean environment is included in the audio signals of the various noise/reverberation environments. Specifically, a large amount of clean speech obtained from a speech corpus, etc. and noise and reverberation data (noise signal waveform, room impulse response, etc.) obtained in various environments are used to observe signals in various environments. The simulated observation signal generated by simulating is input to the case model generation device 200 as a voice signal for learning. The following processing is performed on each of these learning audio signals.

The Fourier transform unit 201 and the feature amount generation unit 202 perform the same processing as that of the Fourier transform unit 102 and the feature amount generation unit 103 in the signal processing device 100 shown in FIG. 1, on the learning audio signal. When the input voice is clean voice, the Fourier transform unit 201 stores the amplitude spectrum of the clean voice in the case model storage unit 101 of the signal processing device 100 as a part of the case model Ms.

The DNN-HMM acoustic model learning unit 203 learns an HMM acoustic model based on DNN using the feature amount generated by the feature amount generating unit 202. The DNN-HMM acoustic model learning unit 203 inputs the feature amount into the HMM acoustic model based on DNN to perform learning, and acquires the likelihood of the HMM state output by the HMM acoustic model based on DNN. The DNN-HMM acoustic model learning unit 203 inputs the feature amount y _t generated by the feature amount generating unit 202 as learning data into the DNN-HMM acoustic model, and the likelihood p of the HMM state output from the DNN-HMM acoustic model. (Y _t |s) is output to the maximum likelihood HMM state calculation unit 204. At this time, the DNN-HMM acoustic model learning unit 203 performs calculation processing using the a priori probability P(s) of the HMM state s in the matching unit 104 of the signal processing device 100, as shown in Expression (2). , A prior probability P(s) of the HMM state s is also generated and stored in the case model storage unit 101 of the signal processing device 100 as a part of the case model.

The maximum likelihood HMM state calculation unit 204 determines, for each time frame, based on the DNN-HMM acoustic model g _s output by the DNN-HMM acoustic model learning unit 203, that is, the likelihood p(y _t |s) of the HMM state. A sequence of s (maximum likelihood HMM state sequence) that is an index of the HMM state that gives the maximum likelihood to the feature amount is calculated. The maximum likelihood HMM state calculation unit 204 obtains an HMM state index sequence (maximum likelihood HMM state sequence) s _i that gives the maximum likelihood for each time frame i, and the obtained time sequence (segment) of the index s _i. Is stored in the case model storage unit 101 of the signal processing device 100 as a case model Ms based on DNN-HMM.

[Case model generation process]
Next, the case model generation process will be described. FIG. 6 is a flowchart showing the processing procedure of the case model generation processing by the case model generation device 200 shown in FIG.

In the case model generation device 200, the Fourier transform unit 201 and the feature amount generation unit 202 perform the processing of steps S11 and S12 on the input learning audio signal in the same procedure as steps S1 and S2 shown in FIG. To do.

The DNN-HMM acoustic model learning unit 203 performs learning processing of the DNN-HMM acoustic model using the feature amount input from the feature amount generating unit 202 in the previous stage (step S13). The DNN-HMM acoustic model learning unit 203 also calculates the prior probability P(s) of the HMM state s.

Subsequently, the maximum likelihood HMM state calculation unit 204 is an HMM state index that gives the maximum likelihood to the feature amount for each time frame, based on the likelihood of the HMM state output by the DNN-HMM acoustic model. Maximum-likelihood HMM state calculation processing for calculating the series of s (maximum-likelihood HMM state series) as the case model Ms is performed (step S14). Then, the case model generation device 200 performs a storage process of storing the maximum likelihood HMM state series in the case model storage unit 101 of the signal processing device 100 as the case model Ms (step S15).

In this way, the case model generation device 200 corresponds to the signal processing device 100 and generates the case model Ms using the DNN-HMM acoustic model. Therefore, the signal processing apparatus 100 can execute the matching process using the case model Ms that reflects high noise resistance.

[Effect of Embodiment]
In the signal processing device 100 according to this embodiment, signal processing is performed using the DNN-HMM acoustic model. This DNN-HMM acoustic model has high noise resistance. Specifically, in the signal processing device 100, the feature amount of the input signal is input to the DNN-HMM acoustic model, and the output result by the DNN-HMM acoustic model and the case model Ms stored in the case model storage unit 101 are displayed. By collating, the clean speech feature amount corresponding to the input signal is obtained.

As described above, the DNN-HMM acoustic model has high noise resistance. In other words, the DNN-HMM acoustic model can estimate the HMM state with high accuracy even if the feature amount of the input speech is affected by noise or reverberation. Therefore, in the signal processing apparatus 100, by using the DNN-HMM acoustic model having high noise resistance, it is possible to perform a segment search that is robust against noise and reverberation, that is, a segment search that is less susceptible to noise and reverberation. become. Further, the case model generation device 200 corresponds to the signal processing device 100 and generates the case model Ms using the DNN-HMM acoustic model. Therefore, the signal processing apparatus 100 can execute the matching process using the case model Ms that reflects high noise resistance.

As described above, according to the present embodiment, it is possible to reduce the influence of noise and reverberation on the search for clean speech similar to the input signal, and it is possible to accurately search for clean speech similar to the input signal.

[Modification]
In the present embodiment, the case where the DNN-HMM acoustic model is used has been described. This DNN-HMM acoustic model is based on the so-called fully-connected feed forward neural network, but in the present embodiment, it is also possible to use an acoustic model based on the Neural Network of other structure. For example, in the present embodiment, for example, an acoustic model based on a Convolutional Neural Network (CNN: convolutional neural network), an acoustic model based on a Recurrent Neural Network (RNN: recursive neural network), or an LSTM (Long Short-Term Memory) is used. It may be an acoustic model of the RNN based on.

In this case, the case model storage unit 101 selects one of the CNN-based acoustic model, the RNN-based acoustic model, and the LSTM-based RNN acoustic model for the speech or clean speech including noise or acoustic distortion. The case model learned by using it is stored. Then, the matching unit 104 inputs the feature amount into any one of the CNN-based acoustic model, the RNN-based acoustic model, and the LSTM-based RNN acoustic model, and outputs the output result by any one of the Neural Networks. , And collates with the case model stored in the case model storage unit 101.

Further, in the above description, the DNN-HMM acoustic model or other acoustic model based on the NN structure is assumed to be learned by the case model generation device, but this is not essential, and is described in Non-Patent Document 2, for example. It is also possible to use an existing acoustic model that has been separately learned by the method in the case model generation device. These acoustic models are not necessarily the above-mentioned simulated observation signals, and may be those learned by a speech signal that originally contains noise and reverberation.

Further, in the present embodiment, matching section 104 may divide the input signal into two and perform matching processing on each of the first half portion and the second half portion. Regarding division of the input signal, refer to, for example, Japanese Patent No. 6139429 by the applicant or Japanese Patent No. 6139430 by the applicant.

[System configuration, etc.]
The respective constituent elements of the illustrated devices are functionally conceptual, and do not necessarily have to be physically configured as illustrated. That is, the specific form of distribution/integration of each device is not limited to the one shown in the figure, and all or part of the device may be functionally or physically distributed/arranged in arbitrary units according to various loads and usage conditions. It can be integrated and configured. For example, the signal processing device 100 and the case model generation device 200 may be an integrated device. Further, each processing function performed by each device may be realized in whole or in an arbitrary part by a CPU and a program analyzed and executed by the CPU, or may be realized as hardware by a wired logic.

Further, of the processes described in the present embodiment, all or part of the processes described as being automatically performed may be manually performed, or the processes described as being manually performed may be performed. The whole or part of the process can be automatically performed by a known method. Further, each processing described in the present embodiment is not only executed in time series according to the order described, but may be executed in parallel or individually according to the processing capacity of the device that executes the processing or the need. .. In addition, the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above-mentioned documents and drawings can be arbitrarily changed unless otherwise specified.

[program]
FIG. 7 is a diagram illustrating an example of a computer in which the signal processing device 100 or the case model generation device 200 is realized by executing a program. The computer 1000 has, for example, a memory 1010 and a CPU 1020. The computer 1000 also has a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These units are connected by a bus 1080.

The memory 1010 includes a ROM 1011 and a RAM 1012. The ROM 1011 stores, for example, a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1031. The disk drive interface 1040 is connected to the disk drive 1041. For example, a removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1041. The serial port interface 1050 is connected to, for example, a mouse 1110 and a keyboard 1120. The video adapter 1060 is connected to the display 1130, for example.

The hard disk drive 1031 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. That is, the program defining each process of the signal processing device 100 or the case model generation device 200 is implemented as a program module 1093 in which a code executable by the computer 1000 is described. The program module 1093 is stored in, for example, the hard disk drive 1031. For example, a program module 1093 for executing the same processing as the functional configuration in the signal processing device 100 or the case model generation device 200 is stored in the hard disk drive 1031. The hard disk drive 1031 may be replaced with an SSD (Solid State Drive).

The setting data used in the processing of the above-described embodiment is stored as the program data 1094 in, for example, the memory 1010 or the hard disk drive 1031. Then, the CPU 1020 reads out the program module 1093 and the program data 1094 stored in the memory 1010 or the hard disk drive 1031 to the RAM 1012 as necessary and executes them.

The program module 1093 and the program data 1094 are not limited to being stored in the hard disk drive 1031 and may be stored in, for example, a removable storage medium and read by the CPU 1020 via the disk drive 1041 or the like. Alternatively, the program module 1093 and the program data 1094 may be stored in another computer connected via a network (LAN (Local Area Network), WAN (Wide Area Network), etc.). Then, the program module 1093 and the program data 1094 may be read by the CPU 1020 from another computer via the network interface 1070.

Although the embodiments to which the invention made by the present inventor has been applied have been described above, the present invention is not limited to the description and the drawings that form part of the disclosure of the present invention according to the present embodiment. That is, all other embodiments, examples, operation techniques, and the like made by those skilled in the art based on this embodiment are included in the scope of the present invention.

100, 100P Signal processing device 101, 101P Case model storage unit 102, 102P Fourier transform unit 103, 103P Feature amount generation unit 104, 104P Matching unit 105, 105P Speech enhancement filtering unit 200, 200P Case model generation device

Claims (8)

A storage unit that stores a case model that is output by using an acoustic model based on Neural Network, with speech or clean speech including noise or acoustic distortion as an input,
A feature amount generation unit that generates a feature amount from an input signal,
The clean speech feature amount corresponding to the input signal is obtained by matching the output result output using the acoustic model based on the Neural Network with the feature amount as an input, and the case model stored in the storage unit. A matching section,
An output unit for outputting a emphasized voice obtained by multiplying the input signal by a filter composed of the clean voice feature amount obtained by the matching unit;
A signal processing device comprising: The said storage part and the said collation part use the HMM(Hidden Markov Model) acoustic model based on DNN(Deep Neural Network) based on the input of the speech containing noise or acoustic distortion, or clean speech. The signal processing device described. The storage unit and the matching unit are any one of a CNN (Convolutional Neural Network)-based acoustic model, an RNN (Recurrent Neural Network)-based acoustic model, and an LSTM (Long Short-Term Memory)-based RNN acoustic model. The signal processing device according to claim 1, wherein one of them is used. A feature amount generation unit that generates a feature amount from an input signal for learning,
A learning unit for learning a DNN-based HMM acoustic model using the feature quantity;
A maximum likelihood HMM state in which a series of HMM state indexes that give the maximum likelihood to the feature amount for each time frame is calculated as a case model based on the likelihood of the HMM state output from the HMM acoustic model based on the DNN. A calculation part,
A case model generation device having: The feature amount of the input signal is input to the DNN-based HMM acoustic model, and the output result of the DNN-based HMM acoustic model and the speech or clean speech including noise or acoustic distortion are learned using the DNN-based HMM acoustic model. A collation device comprising: a collation unit that collates with a case model and obtains a clean speech feature amount corresponding to the input signal. A signal processing device executed by the signal processing device, comprising:
The signal processing device has a storage unit that stores a case model output by using a voice or clean voice including noise or acoustic distortion as an input and using an HMM acoustic model based on DNN,
A feature amount generation step of generating a feature amount from an input signal,
The feature amount is input to a DNN-based HMM acoustic model, an output result of the DNN-based HMM acoustic model is collated with a case model stored in the storage unit, and a clean speech feature amount corresponding to the input signal is obtained. And a matching step for obtaining
An output step of outputting a emphasized voice obtained by multiplying the input signal by a filter composed of the clean voice feature amount obtained in the matching step;
A signal processing method comprising: A signal processing method executed by a case model generation device, comprising:
A feature amount generating step of generating a feature amount from an input signal for learning,
A learning step of learning an HMM acoustic model based on DNN using the feature quantity;
A maximum likelihood HMM state in which a series of HMM state indexes that give the maximum likelihood to the feature amount for each time frame is calculated as a case model based on the likelihood of the HMM state output from the HMM acoustic model based on the DNN. Calculation process,
A signal processing method comprising: A signal for causing a computer to function as any one of the signal processing device according to claim 1, the case model generation device according to claim 4, and the matching device according to claim 5. Processing program.