JP6499095B2 - Signal processing method, signal processing apparatus, and signal processing program - Google Patents

Description

  The present invention relates to a signal processing method, a signal processing device, and a signal processing program.

  Conventionally, in a speech recognition system, a hearing aid, a video conference system, a machine control interface, a music information processing system for searching and recording music, etc., an acoustic signal is collected using a microphone and a component of the target audio signal is extracted. Technology is used.

  In general, when a sound signal is collected using a microphone in a real environment with noise and reverberation, not only a sound signal for sound collection but also a signal on which noise and reverberation (acoustic distortion) are superimposed is observed. However, when these noises and reverberations are superimposed on the signal, it becomes difficult to extract the components of the sound signal for sound collection, which causes a significant reduction in the clarity and ease of hearing of the sound signal. As a result, for example, there is a problem that the recognition rate of the voice recognition system is lowered.

  Therefore, a technique for removing noise and reverberation superimposed on an audio signal has been proposed (see, for example, Non-Patent Document 1). For example, a conventional audio signal processing apparatus will be described with reference to FIG. FIG. 10 is a block diagram showing an example of the configuration of a conventional signal processing apparatus. Note that the signal processing device 1P shown in FIG. 10 uses the case model expressed by a Gaussian Mixture Model (GMM) to check the similarity with the feature value obtained by converting the input speech, and the high similarity The example model that indicates is used as a sound signal candidate for sound collection.

  In this conventional signal processing apparatus 1P, a case model expressed by a mixture distribution model learned in advance is stored in the case model storage unit 11P. Specifically, in the case model storage unit 11P, the Gaussian mixture distribution that gives the maximum likelihood to the amplitude spectrum of clean speech corresponding to each case and the feature amount (for example, Mel frequency cepstrum coefficient) for each frame. A case model including a series (segment) of indexes is stored.

  First, the Fourier transform unit 12P obtains an amplitude spectrum by performing discrete Fourier transform on the input signal including the acoustic distortion, and the feature amount generation unit 13P generates a segment of the feature amount from the amplitude spectrum.

  Subsequently, the matching unit 15P performs matching between the segment of the feature amount generated by the feature amount generation unit 13P and the segment included in the case model of the case model storage unit 11P, and from the case model, the feature amount generation unit 13P The segment having the highest similarity to the segment of the feature amount generated by is searched. Specifically, the matching unit 15P searches the segment of the case model for a segment that gives the maximum posterior probability with respect to the feature amount segment generated by the feature amount generation unit 13P.

  Then, the speech enhancement filtering unit 16P regards the clean speech amplitude spectrum corresponding to the feature quantity of the segment of the case model searched by the matching unit 15P as the clean speech amplitude spectrum most similar to the input signal, and the case model storage unit. From 11P, the amplitude spectrum of the clean speech is read out to create a filter for speech enhancement. By filtering the input signal with this filter, an enhanced speech signal from which acoustic distortion has been removed from the input signal is obtained.

J. Ming and R. Srinivasan, and D. Crookes, “A Corpus-Based Approach to Speech Enhancement From Nonstationary Noise,” IEEE Transactions on Audio, Speech, and Language Processing, Vol.19, No.4, pp.822- 836, 2011

  As described above, the conventional signal processing device 1P uses the feature amount segment generated by the feature amount generation unit 13P in order to obtain the amplitude spectrum of the clean sound most similar to the input sound. A segment that gives the maximum posterior probability is searched from the case model.

  However, the mel frequency cepstrum coefficient used for the segment search is a simple feature amount obtained from the amplitude spectrum. For this reason, when noise and reverberation are included in the input signal, the mel frequency cepstrum coefficient also includes the influence of noise and reverberation, and the segment search by the matching unit 15P is not necessarily highly accurate.

  In addition, although the case model is prepared assuming various acoustic distortion environments, it is actually difficult to prepare case models corresponding to all the acoustic distortion environments. In some cases, a segment having a high degree of similarity with the feature amount segment generated by the generation unit 13P cannot be searched from the case model.

  Therefore, in the conventional signal processing apparatus, since the feature amount used for the search is affected by noise and reverberation, there is a limit to the accuracy of searching for the clean speech feature amount similar to the input signal.

  The present invention has been made in view of the above, and it is an object of the present invention to provide a signal processing method, a signal processing apparatus, and a signal processing program that reduce the influence of noise and reverberation on a search for clean speech similar to an input signal. And

  In order to solve the above-described problems and achieve the object, a signal processing method according to the present invention is a signal processing method executed by a signal processing device, and the signal processing device is a voice including noise or acoustic distortion. Or it has a storage part which memorizes the mixture distribution model which learned clean speech, the signal processing device generates the 1st feature amount from an input signal, and the signal processing device has the 1st above-mentioned A feature amount conversion step of converting the feature amount of the second feature amount into a second feature amount subjected to noise or acoustic distortion reduction processing, and the signal processing device based on the parameters of the mixed distribution model stored in the storage unit. In addition, a posterior probability indicating the probability that the second feature amount corresponds to each distribution of the mixed distribution model is calculated, and a clean speech feature amount having the highest posterior probability is defined as a clean speech feature amount corresponding to the input signal. A matching step, and the signal processing device includes an output step of outputting an enhanced speech signal obtained by multiplying the input signal by a filter composed of the clean speech feature obtained in the matching step. Features.

  According to the present invention, it is possible to reduce the influence of noise and reverberation on a search for clean speech similar to an input signal.

FIG. 1 is a diagram schematically illustrating an example of the configuration of the signal processing device according to the first embodiment. FIG. 2 is a diagram for explaining an example of a segment. FIG. 3 is a conceptual diagram for explaining processing of the feature amount conversion unit shown in FIG. FIG. 4 is a flowchart showing a processing procedure executed by the signal processing apparatus shown in FIG. FIG. 5 is a block diagram illustrating a functional configuration example of the case model generation apparatus according to the first embodiment. FIG. 6 is a flowchart showing a processing procedure of case model generation processing by the case model generation apparatus shown in FIG. FIG. 7 is a block diagram showing the configuration of the signal processing apparatus according to the second embodiment. FIG. 8 is a flowchart showing a processing procedure executed by the signal processing apparatus shown in FIG. FIG. 9 is a diagram illustrating an example of a computer in which a signal processing apparatus is realized by executing a program. FIG. 10 is a block diagram showing an example of the configuration of a conventional signal processing apparatus.

  Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. In addition, this invention is not limited by this embodiment. Moreover, in description of drawing, the same code | symbol is attached | subjected and shown to the same part.

[Embodiment 1]
First, the signal processing apparatus according to the first embodiment will be described. This signal processing device is a device that performs processing for removing acoustic distortion from an input signal including noise and reverberation (acoustic distortion) and outputting a clear enhanced speech signal.

[Configuration of signal processing apparatus]
FIG. 1 is a diagram schematically illustrating an example of the configuration of the signal processing device according to the first embodiment. For example, the signal processing apparatus 1 according to the first embodiment reads a predetermined program into a computer or the like including, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), a CPU (Central Processing Unit), and the like. Is realized by executing a predetermined program.

  As shown in FIG. 1, the signal processing apparatus 1 includes a case model storage unit 11, a Fourier transform unit 12, a feature amount generation unit 13, a feature amount conversion unit 14, a matching unit 15 (collation unit), and a speech enhancement filtering unit 16 ( Output section). The signal processing apparatus 1 uses the case model M expressed by the GMM to check the similarity with the feature amount obtained by converting the input signal, and uses the case model M showing a high similarity as a speech signal candidate for sound collection purposes. Use.

  The case model storage unit 11 stores a mixed distribution model obtained by learning speech including acoustic distortion or clean speech. Specifically, the case model storage unit 11 stores clean voice data corresponding to a case and a case model M. The clean voice data is, for example, the amplitude spectrum of clean voice corresponding to the case. In addition, the case model M includes, as a parameter of the mixed distribution model, a sequence (segment) of a Gaussian mixed distribution index that gives the maximum likelihood to the feature amount for each frame.

  Here, the case model M is generated in advance by the case model generation device 2 (described later) and stored in the case model storage unit 11. The example model generation device 2 uses a large amount of clean speech obtained from a speech corpus and noise and reverberation data (noise signal waveform, indoor impulse response, etc.) obtained in various environments, and in various environments. A case model M is generated by using the observation signal that is simulated and generated as a speech signal for learning, and the simulated observation signal is converted into a feature amount region.

Specifically, the Gaussian distribution in the Gaussian mixture model g that gives the maximum likelihood for each time frame i based on the feature amount of the speech signal for learning by the case model generation device 2 (described later). index m _i is determined, the sought time sequence index m _i (segment), and one case model M. This case model M is expressed as shown in the following (1) equation by using the set and Gaussian mixture model g of the index m _i of the Gaussian distribution in the Gaussian mixture model g.

Incidentally, m _i is the index of the Gaussian distribution that gives the maximum likelihood for the feature amount k _i of i-th frame, Gaussian g in Gaussian mixture m _| represents (k i _m) Yes. I represents the total number of frames of the speech signal for learning. For example, assuming 1 hour of learning data, I = 3.5 × 10 ⁵ .

An example of segments included in the case model M will be described. FIG. 2 is a diagram for explaining an example of a segment. For example, each cell of the segment shown in FIG. 2 corresponds to the i-th time frame of the I frame. The numbers in each cell represents the index m _i of the Gaussian distribution of the Gaussian mixed model g that gives the maximum likelihood.

  The Fourier transform unit 12 converts the input signal into an amplitude spectrum for each frame. An audio signal including noise and reverberation is input to the Fourier transform unit 12 as this input signal. First, the Fourier transform unit 12 cuts out waveform data of an input signal with a short time width. For example, the Fourier transform unit 12 cuts out an input signal with a short time width by multiplying a window function such as a short-time Hamming window of about 30 (msec). Subsequently, the Fourier transform unit 12 performs a discrete Fourier transform process on the extracted input signal to convert it into an amplitude spectrum. The amplitude spectrum is amplitude data of the frequency spectrum. The Fourier transform unit 12 inputs the converted amplitude spectrum to the feature value generation unit 13 and the speech enhancement filtering unit 16.

The feature quantity generation unit 13 generates a feature quantity (first feature quantity) x _t from the amplitude spectrum output from the Fourier transform unit 12. In other words, the feature quantity generation unit 13 generates a segment of the feature quantity x _t from the amplitude spectrum input from the Fourier transform unit 12. Note that t is a frame to be processed. The feature value generation unit 13 converts all of the amplitude spectrum output from the Fourier transform unit 12 into, for example, a mel frequency cepstrum coefficient. As a result, the input signal is represented as a segment of a feature vector for each frame.

Here, the mel frequency cepstrum coefficient generally used is about 10 to 20th order. In the signal processing device 1, in order to accurately represent the case model M, a mel frequency cepstrum coefficient having a higher order (for example, about 30 to 100th order) than a generally used order is used. For this reason, the feature quantity generation unit 13 converts all of the amplitude spectrum output from the Fourier transform unit 12 into, for example, a mel frequency cepstrum coefficient of about 30 to 100th order. Note that the feature quantity generation unit 13 may use a feature quantity (for example, a cepstrum coefficient) other than the mel frequency cepstrum coefficient. The feature quantity generation unit 13 inputs the generated feature quantity _xt to the feature quantity conversion unit 14.

The feature amount conversion unit 14 converts the feature amount x _t generated by the feature amount generation unit 13 into a feature amount (second feature amount) subjected to noise or reverberation (acoustic distortion) reduction processing. That is, the feature amount conversion unit 14 converts the feature amount generated by the feature amount generation unit 13 such as a mel frequency cepstrum coefficient into a feature amount having high acoustic distortion resistance.

Specifically, the feature amount conversion unit 14 uses the feature amount x _t generated by the feature amount generation unit 13 in a DNN (Deep Neural Network) -HMM (Hidden Markov Model) acoustic model. A bottleneck feature quantity b _t having high acoustic distortion resistance is generated by applying nonlinear feature quantity transformation in multiple stages. In this case, the feature amount conversion unit 14 generates the bottleneck feature amount b _t by using not only the feature amount segment of the processing target frame but also the feature amount segments of a predetermined number of frames before and after the feature amount segment. The bottleneck feature amount b _t is extracted from a network having a bottleneck structure in which the number of intermediate layer units of the neural network is reduced. The feature quantity extracted in the intermediate layer of the bottleneck structure is a feature quantity having acoustic distortion resistance obtained by dimension-compressing the input feature quantity. The feature amount conversion unit 14 inputs the generated bottleneck feature amount b _t to the matching unit 15.

  Note that the “characteristic amount having acoustic distortion resistance” is an input in which two different acoustic distortions are added, for example, assuming that two different acoustic distortions are added to the same input voice. Two feature values generated for speech are "similar". In other words, the “characteristic amount having acoustic distortion resistance” is a characteristic quantity in which the influence of the acoustic distortion is reduced.

The matching unit 15 uses the case model M to check the similarity with the feature amount of the input speech that has been input, and uses the clean speech corresponding to the case model M showing a high similarity as a speech signal candidate for sound collection. Go. Specifically, the matching unit 15 uses the input feature value (the bottleneck feature value b _t converted by the feature value conversion unit 14) based on the parameters of the mixed distribution model stored in the case model storage unit 11. A posteriori probability indicating the probability corresponding to each distribution of the mixed distribution model is calculated, and a clean speech feature value having the highest posterior probability is obtained as a clean speech feature value corresponding to the input signal.

In other words, the matching unit 15 performs matching between the segment of the feature amount (bottleneck feature amount b _t ) input from the feature amount conversion unit 14 and the segment included in the case model M of the case model storage unit 11, and The segment having the highest posterior probability with respect to the input feature amount segment is searched from the case model M in the model storage unit 11. The matching unit 15 inputs information about the segments in the case model M found by the search to the speech enhancement filtering unit 16. Details of the processing of the matching unit 15 will be described later.

  The speech enhancement filtering unit 16 outputs an enhanced speech signal obtained by multiplying the input signal by a filter composed of clean speech feature values obtained by the matching unit 15. Specifically, the speech enhancement filtering unit 16 regards the clean speech amplitude spectrum corresponding to the feature amount of the segment of the case model M searched by the matching unit 15 as the clean speech amplitude spectrum most similar to the input signal, The amplitude spectrum of this clean speech is read from the case model storage unit 11. Subsequently, the speech enhancement filtering unit 16 creates a filter for speech enhancement using the read amplitude spectrum of the clean speech, and filters the input signal using the filter. As a result, the enhanced speech signal from which the acoustic distortion has been removed from the input signal is output from the speech enhancement filtering unit 16.

[Processing of feature quantity conversion unit]
Next, the process of the feature amount conversion unit 14 will be described in detail. Feature transformation unit 14, generated by the feature amount generating unit 13, for example, the feature amounts such as mel-frequency cepstrum coefficient, sound distortion resistance is converted into a high bottleneck feature quantity b _t. As described above, the DNN-HMM acoustic model is applied to the feature amount conversion unit 14. Therefore, with reference to FIG. 3, the process of the feature amount conversion unit 14 will be described.

FIG. 3 is a conceptual diagram for explaining the processing of the feature quantity conversion unit 14 configured using the DNN-HMM acoustic model. The feature quantity conversion unit 14 receives the feature quantity x _t generated by the feature quantity generation unit 13 such as a mel frequency cepstrum coefficient as input data. At this time, the feature amount conversion unit 14 receives not only the feature amount x _t of the processing target frame _t but also the feature amounts of several frames before and after the feature amount x _t .

For example, in addition to the 40-dimensional feature amount x _t (row vector) of the frame t, the feature amount conversion unit 14 includes feature amounts x _t−5 , x _t−4 , x _t−3 , x _t−3 , _xt-2 , _xt-1 , _{xt + 1} , _{xt + 2} , _{xt + 3} , _{xt + 4} , and _{xt + 5} are received. In this case, feature transformation unit 14, feature amount 440 D total 11 frames _{[x t-5 ^ T,} ···, x t ^ T, ···, x t + 5 ^ T] ^ T (T Represents a transpose of a vector).

The feature amount x _t of frame t to be treated, not only the static characteristic quantities, for example, the primary, there is a case composed of two regression coefficients. In this case, the number of dimensions of the feature quantity received by the feature quantity conversion unit 14 also increases. For example, if _xt is composed of a static 40-dimensional feature quantity and its primary and secondary regression coefficients, the total number of dimensions is 120. Considering 5 frames before and after this, the number of dimensions of the feature quantity received by the feature quantity conversion unit 14 is 1320 dimensions.

Subsequently, the feature amount conversion unit 14 that has received the 440-dimensional feature amount passes this through several intermediate layers (typically about 5 to 10 layers) of, for example, 2048 nodes according to the DNN-HMM acoustic model, Finally, for example, a bottleneck feature quantity b _t dimensionally compressed to about 80 dimensions by a bottleneck layer of 80 nodes is acquired. The feature amount conversion unit 14 inputs the bottleneck feature amount b _t to the matching unit 15.

Feature transformation unit 14, by applying a nonlinear feature transformation in multiple stages in DNN-HMM acoustic model may sound distortion resistance obtain high bottleneck feature quantity b _t. The matching unit 15 can perform a segment search with high accuracy by performing a segment search using the bottleneck feature amount b _t input from the feature amount conversion unit 14. Therefore, the processing of the matching unit 15 using this bottleneck feature quantity b _t will be described.

[Processing of matching part]
Here, for simplification of explanation, only an example model M of a certain noise / reverberation environment is considered. Further, for simplification of explanation, and is not considered time warping during the feature amount of the segment y _t and matching training data segment of the input signal. In the first embodiment, the bottleneck feature value b _t converted by the feature value conversion unit 14 is input from the previous feature value conversion unit 14 as the segment y _t of the feature value of the input signal.

First, the matching section 15 performs matching between a segment of the stored cases model M on the segment y _t and case model storage unit 11 of the input feature quantity. Next, the matching unit 15 searches the segment of the case model M for the segment closest to the sequence y _{t: t + τ} of the feature quantity of the input signal, and the clean speech sequence most similar to the clean speech included in the input signal segment seems to give ^M _{t u:} seeking _{u + .tau.max,} outputs. This can be formulated as equation (2).

Here, it is assumed that the input feature quantity y _t is composed of L time frames, and the feature quantity series of the input signal is y = {y _t : t = 1, 2,..., L}. Also, let _{yt: t + τ be} a sequence from the time frame t to t + τ of the feature quantity of the input signal. Then, M _{u: u + τ} = {g, m _i : i = u, u + 1,..., U + τ} is a Gaussian distribution sequence corresponding to continuous time frames from u-th to u + τ-th in the case model M. And

Series y _t of the feature amount of the input _signal: definition and of the distance between a segment in the _{t + tau} and case model M, feature amount sequence y _t of the input _signal: a method of searching for _{t + tau} and closest case model M, Euclid Several other methods can be considered, such as distance. Here, it is assumed that the segment of the case model M closest to the feature quantity series of the input signal has the longest length among the segments of the case model M that closely match the feature quantity series of the input signal. That is, the segment M ^{t u} closest case model M to the feature amount sequence of the input _{signal: u + tau} can be determined by maximizing a posterior probability shown in (3) below.

In this case, p (M _{u: u + τ} | y _{t: t + τ} ) represents the posterior probability, and if y _{t: t + τ} and M _{u: u + τ} are relatively well matched, τ The longer the is, the higher the posterior probability. By taking a method of searching for a longer segment, it is unlikely to be affected by noise that exists locally at a certain time, and it seems that relatively robust matching is performed against noise.

Note that the numerator term p (y _{t: t + τ} | M _{u: u + τ} ) in the equation (3) is the y _{t: t + τ} for the segment of the case model M corresponding to M _{u: u + τ} . Likelihood. This likelihood is calculated by equation (4).

Here, for simplicity, it is assumed that adjacent frames are independent. The first term of the denominator of equation (3) takes the sum of p (y _{t: t + τ} | M _{u ′: u ′ + τ} ) starting from any time frame u ′ in the case model M. It is a thing. The second term of the denominator of the equation (3) is the likelihood of yt _{: t + τ} with respect to the Gaussian mixture model g and is calculated by the equation (5).

Subsequently, the procedure of the segment search process in the matching unit 15 will be described more specifically. First, the maximum segment length is limited to (τ _lim +1) frames. For example, if the maximum segment length is limited to 30 frames, τ _lim = 29.

  First, the matching unit 15 searches for a segment having a segment length = 1 that gives the maximum posterior probability according to the equation (3), with τ = 0, that is, the segment length = 1, under this restriction. Next, the matching unit 15 searches for a segment with segment length = 2 that gives the maximum posterior probability according to the equation (3), with τ = 1, that is, segment length = 2.

The matching unit 15 repeats this process until τ = τ _lim . Then, the matching unit 15 finds a segment that gives the maximum posterior probability from the searched segment candidates having different lengths. τ _max is the length of the segment giving this maximum posterior probability. Such segment search processing in the matching unit 15 can be performed on a case model M that can be expressed by a linear memory for I frames as shown in FIG.

Then, the matching section 15, the segment which gives the maximum a posteriori probability searched, i.e., segment M ^{t u} case model M seems to provide a clean speech sequence most similar to the clean speech included in the input _signal: information about _{u + .tau.max} Is input to the speech enhancement filtering unit 16. Thereby, the speech enhancement filtering unit 16, segment M ^{t _u:} by using the amplitude spectrum of the clean speech in the case the model storage unit 11 corresponding to the _{u + .tau.max,} to create a filter for the speech enhancement, the input signal at the filter Is output as an enhanced speech signal.

[Signal processing method in signal processing apparatus]
Next, a signal processing method in the signal processing apparatus 1 will be described. FIG. 4 is a flowchart showing a processing procedure executed by the signal processing device 1 shown in FIG.

  First, the Fourier transform unit 12 performs a Fourier transform process (step S1) for converting an input signal into an amplitude spectrum. The feature amount generation unit 13 performs a feature amount generation process (step S2) for generating a feature amount such as a mel frequency cepstrum coefficient from the amplitude spectrum output from the Fourier transform unit 12.

  The feature amount conversion unit 14 performs a feature amount conversion process (step S3) for converting the feature amount generated by the feature amount generation unit 13 into a bottleneck feature amount subjected to noise or reverberation (acoustic distortion) reduction processing.

  The matching unit 15 performs matching between the segment of the case model M in the case model storage unit 11 and the segment of the input bottleneck feature amount, and from the segment of the case model M, the input bottleneck feature amount A matching process (step S4) for searching for a segment that takes a segment having the highest posterior probability is performed.

  The speech enhancement filtering unit 16 creates a filter for speech enhancement using the amplitude spectrum of clean speech corresponding to the feature amount of the segment of the case model M searched by the matching unit 15, and multiplies the input signal by the filter. A voice enhancement filtering process (step S5) for outputting the emphasized voice is performed.

[Effect of the first embodiment]
Thus, the signal processing apparatus 1 according to the first embodiment, as the characteristic amount y _t the matching section 15 is used to segment the search, rather than simply feature amount x _t is obtained from the amplitude spectrum, such as Mel Frequency Cepstral Coefficients for the feature amount x _t, it is used further noise or reverberation bottleneck characteristic quantity b _t subjected to reduction processing (acoustic distortion). In other words, the matching unit 15, for performing segment search using acoustic distortion resistance is high bottleneck characteristic quantity b _t, can reduce the influence of noise or reverberation for the segment search, it is possible to improve the accuracy of the segment search. Therefore, according to the signal processing device 1, it is possible to search for a feature amount of clean speech similar to the input signal with high accuracy, and to convert the input signal into a clear enhanced speech signal.

[Case model generator]
The case model generation device 2 that generates the case model M stored in the case model storage unit 11 of the signal processing device 1 will be described. In this case the model generating apparatus 2, for example, a bottleneck features against the feature amount x _t such mel-frequency cepstrum coefficients generated from the speech signals for learning, subjected to reduction processing of noise or reverberation (acoustic distortion) A case model M is generated using the quantity b _t .

  FIG. 5 is a block diagram illustrating a functional configuration example of the case model generation device 2. The case model generation device 2 illustrated in FIG. 5 is realized by, for example, a predetermined program being read into a computer including a ROM, a RAM, a CPU, and the like, and the CPU executing the predetermined program. The case model generation apparatus 2 includes a Fourier transform unit 12, a feature amount generation unit 13, a feature amount conversion unit 14, a Gaussian mixture model learning unit 25, and a maximum likelihood Gaussian distribution calculation unit 26.

  First, a learning speech signal input to the case model generation device 2 will be described. The signal input to the case model generation apparatus 2 is an audio signal having various noise / reverberation environments. Among the various noise / reverberation environment audio signals, clean environment audio signals are included. Specifically, using a large amount of clean speech obtained from a speech corpus and noise and reverberation data (noise signal waveforms, room impulse responses, etc.) obtained in various environments, observation signals in various environments The simulated observation signal generated by simulating the signal is input to the case model generation device 2 as a speech signal for learning. The following processing is performed for each of these learning speech signals.

The Fourier transform unit 12, the feature amount generation unit 13, and the feature amount conversion unit 14 perform the same processes as the Fourier transform unit 12, the feature amount generation unit 13, and the feature amount conversion unit 14 in the signal processing apparatus 1 illustrated in FIG. This is performed on the audio signal for learning. The feature amount conversion unit 14 converts the feature amount x _t corresponding to the learning speech signal into a bottleneck feature amount b _t and inputs it to the Gaussian mixture model learning unit 25.

Gaussian mixture model learning unit 25, as learning data characteristic quantity b _i for each short time frame t, obtaining a Gaussian mixture model g by a conventional maximum likelihood estimation. Here, the Gaussian mixture model learning unit 25 obtains a Gaussian mixture model g by using the bottleneck feature amount b _t input from the preceding feature amount conversion unit 14 as learning data. This Gaussian mixture model g is expressed by equation (6). Further, g (b _i | m) representing the Gaussian distribution in the Gaussian mixture model g is expressed by the equation (7). Note that b _i is the bottleneck feature amount of the i-th frame.

g (b _i | m) represents the m-th Gaussian distribution with mean μ _m and variance Σ _m . g (b _i | m) is often a multidimensional Gaussian distribution, and the number of dimensions is the same as the number of dimensions of the feature quantity b _i . When g (b _i | m) is a multidimensional Gaussian distribution, each of the mean μ _m and the variance Σ _m is a vector. Here, even if g (b _i | m) is a multidimensional Gaussian distribution, g (b _i | m) is simply expressed as a Gaussian distribution in order to simplify the description. w (m) represents the mixing weight for the mth Gaussian distribution. Q represents the number of mixtures. For Q, for example, a fairly large value such as 4096 or 8192 is set.

Maximum likelihood Gaussian distribution calculation unit 26 calculates an index m _i of the Gaussian distribution in the Gaussian mixture model g that gives the maximum likelihood for each time frame i, the time sequence of the index m _i, case model M As a single segment. Incidentally, case model M, using the set and Gaussian mixture model g of the index m _i of the Gaussian distribution is expressed as previously described (1).

  The generation of the segment of the case model M is performed for each of the learning speech signals, and the generated case model M including each segment is stored in the case model storage unit 11 (FIG. 1). When the environment is clean, the amplitude spectrum data output from the Fourier transform unit 12 is also stored in the case model storage unit 11 (FIG. 1) as the amplitude spectrum of clean speech.

[Case model generation process]
Next, case model generation processing will be described. FIG. 6 is a flowchart showing a processing procedure of case model generation processing by the case model generation device 2.

  In the example model generation device 2, the Fourier transform unit 12, the feature amount generation unit 13, and the feature amount conversion unit 14 perform steps in the same procedure as steps S1 to S3 shown in FIG. 4 for the input learning speech signal. The process of S11-step S13 is performed.

The Gaussian mixture model learning unit 25 performs a Gaussian mixture model learning process for obtaining a Gaussian mixture model g by a normal maximum likelihood estimation method using the bottleneck feature amount b _t input from the preceding feature amount conversion unit 14 as learning data. Perform (step S14).

Subsequently, the maximum likelihood Gaussian distribution calculation unit 26 calculates an index m _i of the Gaussian distribution in the Gaussian mixture model g that gives the maximum likelihood for each time frame i, the time sequence of the index m _i obtained Then, the maximum likelihood Gaussian distribution calculation process acquired as one segment of the case model M is performed (step S15). The case model generating device 2, the time sequence of the index m _i, performs a storage process for storing as one segment case model M in case the model storage unit 11 of the signal processing apparatus 1 (step S16).

As described above, the case model generation device 2 generates the case model M using the bottleneck feature quantity b _t in correspondence with the signal processing device 1.

[Embodiment 2]
Next, a second embodiment will be described. In the second embodiment, a signal processing device that performs a segment search in consideration of speaker characteristics while reducing the influence of acoustic distortion will be described.

[Configuration of signal processing apparatus]
FIG. 7 is a block diagram showing the configuration of the signal processing apparatus according to the second embodiment. As shown in FIG. 7, the signal processing device 201 according to the second embodiment is a speaker feature value generation unit 217 provided in parallel with the feature value conversion unit 14 as compared with the signal processing device 1 shown in FIG. 1. And a connection unit 218 provided at the subsequent stage of the feature amount conversion unit 14 and the speaker feature amount generation unit 217.

The speaker feature value generation unit 217 generates a speaker feature value expressing the features of the speaker. The speaker feature quantity generation unit 217 receives a feature quantity x _t such as a mel cepstrum output from the feature quantity generation unit 13, and uses the feature quantity x _t to express speaker characteristics such as an i-vector. A speaker feature w _t of about several tens to several hundreds of dimensions is generated.

The concatenating unit 218 connects the bottleneck feature value b _t converted by the feature value converting unit 14 and the speaker feature value w _t generated by the speaker feature value generating unit 217 [b _t ^ T, w _t ^ T] ^ T (T represents transposition of the vector) is generated and input to the matching unit 15 at the subsequent stage.

Then, the matching unit 15 calculates a posteriori probability indicating the probability that the connected feature value [b _t ^ T, w _t ^ T] ^ T corresponds to each distribution of the mixed distribution model, and clean speech that takes the highest a posteriori probability. The feature amount is obtained as a clean speech feature amount corresponding to the input signal.

[Processing of speaker feature generator]
Here, the generation process of the speaker feature quantity w _t by the speaker feature quantity generation unit 217 will be described. Here, the speaker feature amount generating unit 217, will be described for generating a feature vector called i-vector expressed in a vector of several tens to several hundreds dimensional characteristics of the speaker (vector w _e). Here, a method for extracting an i-vector in speaker recognition using a GMM-UBM (Universal Background Model) approach will be described. In the GMM-UBM approach, a “voice-like” model (UBM) is learned using a large amount of speech data for UBM learning of a large number of unspecified speakers, and a new speaker model (GMM) This is a technique of adaptively obtaining UBM using a small amount of voice data of a speaker. The UBM is stored in a storage unit (not shown).

Hereinafter, we describe a specific series of extraction steps in an i-vector vector w _e. To determine the i-vectorw _e, first, by using the equation (3) shown in the first embodiment, each of the feature vector series X _e of L frames obtained from the input signal e that is input to the signal processing unit 201 A posteriori probability γ _t (m) in which the feature value x _t (t = 1, 2,..., L) of the frame is generated from the mth Gaussian distribution of the UBM is calculated. Subsequently, (3) using the posterior probability gamma _t (m) is calculated by the formula, according to (8) to (12) below to calculate the i-vectorw _e.

Using the posterior probability γ _t (m), the 0th-order and 1st-order Baum-Welch statistics N _{e, m} and the vector F _{e, m} for the input signal e using UBM are expressed by the following equation (8) and ( 9) Each can be written as However, the vector F _{e, m} is a D-dimensional vector.

Further, using equation (8) and equation (9), a matrix N _e and a vector F _e that are 0th-order and first-order Baum-Welch statistics are defined as in equations (10) and (11). To do. However, the matrix N _e is a CD dimension × CD dimension matrix, and the vector F _e is a D dimension vector.

Here, the matrix _ID that appears in the diagonal component of the above equation (10) is a D-dimensional × D-dimensional unit matrix. The matrix T is a CD dimension × M dimension rectangular matrix (M << CD) called a total variation matrix. The matrix Σ is a D-dimensional D-dimensional diagonal covariance matrix that models residual fluctuation components that cannot be expressed by the total fluctuation matrix T. I-vectorw _e using the above can be calculated as (12).

Note that (12) matrix _{I M} in formula is a unit matrix of the M-dimensional × M dimension. (12) a i-vector of M dimension for vector w _e is input voice data e in the expression. Speaker feature amount generating unit 217, the vector _{w e,} as the speaker characteristic quantity _{w t,} and outputs the coupling portion 218.

[Processing of signal processor]
Therefore, a process until the signal processing apparatus 201 outputs an enhanced audio signal will be described.
FIG. 8 is a flowchart showing a processing procedure executed by the signal processing device 201.

Steps S21 to S23 are steps S1 to S3 shown in FIG. The speaker feature amount generating unit 217, by using the input feature amount x _t, performs speaker feature quantity generation process for generating a speaker characteristic quantity w _t (step S24). Note that step S23 and step S24 are executed in parallel, for example.

The concatenating unit 218 connects the bottleneck feature value b _t converted by the feature value converting unit 14 and the speaker feature value w _t generated by the speaker feature value generating unit 217 [b _t ^ T, A concatenation process for generating w _t ^ T] ^ T (T represents transposition of the vector) is performed (step S25).

The matching unit 15 uses the connected feature [b _t ^ T, w _t ^ T] ^ T generated by the connecting unit 218 as a matching target for the segment of the case model M in the case model storage unit 11, and performs the steps of FIG. The same processing procedure as S4 is performed to perform matching processing (step S26). Step S27 shown in FIG. 8 is step S5 shown in FIG.

[Effect of Embodiment 2]
In speech recognition, speaker characteristics are unnecessary information. Therefore, in the feature value conversion processing through the DNN-HMM acoustic model, feature value conversion that reduces speaker characteristics is performed. Therefore, when the feature amount conversion unit 14 extracts the bottleneck feature amount through the DNN-HMM acoustic model, the speaker characteristic is also reduced. Therefore, in the second embodiment, a segment search is performed by the matching unit 15 using a connected feature value obtained by connecting the speaker feature value w _t to the bottleneck feature value b _t with reduced speaker characteristics, thereby obtaining a final result. In particular, the emphasized speech signal output from the signal processing device 201 may include speaker characteristics.

  As described above, in the second embodiment, the matching unit 15 uses the influence of the acoustic distortion by connecting the bottleneck feature quantity that reduces the influence of the acoustic distortion and the speaker feature quantity that expresses the speaker characteristics. This makes it possible to perform segment search in consideration of speaker characteristics.

[Configuration of Signal Processing Device and Case Model Generation Device]
In the present invention, the time when considering a case model of a plurality of acoustic distortions (noise / reverberation environment) and the time when considering time expansion and contraction at the time of matching are described in Non-Patent Document 1. It is extensible. Further, the case model storage unit 11 may store, for example, a case model M including a segment to which a tree structured configuration described in Japanese Patent Application Laid-Open No. 2015-152704 by the applicant is applied. In this case, the matching unit 15 may search for a segment most similar to the segment corresponding to the input signal from the case model M including the segment having the tree structure. Moreover, the matching part 15 may perform a segment search, for example using the segment evaluation function described in Unexamined-Japanese-Patent No. 2015-152705 by the applicant.

[System configuration, etc.]
Each component of each illustrated device is functionally conceptual and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to the one shown in the figure, and all or a part of the distribution / integration is functionally or physically distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured. For example, the signal processing devices 1 and 201 and the case model generation device 2 may be an integrated device. Further, all or a part of each processing function performed in each device can be realized by a CPU and a program that is analyzed and executed by the CPU, or can be realized as hardware by wired logic.

  In addition, among the processes described in this embodiment, all or a part of the processes described as being automatically performed can be manually performed, or the processes described as being manually performed can be performed. All or a part can be automatically performed by a known method. In addition, each process described in the present embodiment is not only executed in time series according to the order of description, but may be executed in parallel or individually as required by the processing capability of the apparatus that executes the process. . In addition, the processing procedure, control procedure, specific name, and information including various data and parameters shown in the above-described document and drawings can be arbitrarily changed unless otherwise specified.

[program]
FIG. 9 is a diagram illustrating an example of a computer in which a signal processing device or a learning model generation device is realized by executing a program. The computer 1000 includes a memory 1010 and a CPU 1020, for example. The computer 1000 also includes a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These units are connected by a bus 1080.

  The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM 1012. The ROM 1011 stores a boot program such as BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1031. The disk drive interface 1040 is connected to the disk drive 1041. For example, a removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1041. The serial port interface 1050 is connected to a mouse 1110 and a keyboard 1120, for example. The video adapter 1060 is connected to the display 1130, for example.

  The hard disk drive 1031 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. That is, a program that defines each process of the signal processing device and the learning model generation device is implemented as a program module 1093 in which a code executable by the computer 1000 is described. The program module 1093 is stored in the hard disk drive 1031, for example. For example, a program module 1093 for executing processing similar to the functional configuration in the signal processing device and the learning model generation device is stored in the hard disk drive 1031. The hard disk drive 1031 may be replaced by an SSD (Solid State Drive).

  The setting data used in the processing of the above-described embodiment is stored as program data 1094 in, for example, the memory 1010 or the hard disk drive 1031. Then, the CPU 1020 reads the program module 1093 and the program data 1094 stored in the memory 1010 and the hard disk drive 1031 to the RAM 1012 as necessary, and executes them.

  Note that the program module 1093 and the program data 1094 are not limited to being stored in the hard disk drive 1031, but may be stored in, for example, a removable storage medium and read out by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and the program data 1094 may be stored in another computer connected via a network (LAN, WAN, etc.). Then, the program module 1093 and the program data 1094 may be read by the CPU 1020 from another computer via the network interface 1070.

  As mentioned above, although embodiment which applied the invention made | formed by this inventor was described, this invention is not limited with the description and drawing which make a part of indication of this invention by this embodiment. That is, other embodiments, examples, operation techniques, and the like made by those skilled in the art based on this embodiment are all included in the scope of the present invention.

1, 1P, 201 Signal processing device 2 Case model generation device 11, 11P Case model storage unit 12, 12P Fourier transform unit 13, 13P Feature amount generation unit 14 Feature amount conversion unit 15, 15P Matching unit 16, 16P Speech enhancement filtering unit 25 Gaussian mixture model learning unit 26 Maximum likelihood Gaussian distribution calculation unit 217 Speaker feature generation unit 218 Connection unit

Claims (5)

A signal processing method executed by a signal processing device,
The signal processing apparatus includes a storage unit that stores a mixed distribution model in which a voice including noise or acoustic distortion or a clean voice is learned,
A feature amount generating step in which the signal processing device generates a first feature amount from an input signal;
A feature amount conversion step in which the signal processing device converts the first feature amount into a second feature amount subjected to noise or acoustic distortion reduction processing;
A speaker feature generating step in which the signal processing device generates a speaker feature that expresses the feature of the speaker;
A connecting step in which the signal processing device generates a connected feature value obtained by connecting the second feature value and the speaker feature value;
The signal processing device calculates a posterior probability indicating the probability that the connected feature value corresponds to each distribution of the mixed distribution model based on the parameters of the mixed distribution model stored in the storage unit, and has the highest posterior A collation step for obtaining a clean speech feature value taking a probability as a clean speech feature value corresponding to the input signal;
An output step in which the signal processing device outputs an enhanced speech signal obtained by multiplying the input signal by a filter configured from the clean speech feature value obtained in the matching step;
A signal processing method comprising:   2. The signal according to claim 1, wherein the reduction process is a process of obtaining a bottleneck feature amount from a DNN (Deep Neural Network) -HMM (Hidden Markov Model) acoustic model. Processing method. A learning feature value generation step in which the signal processing device generates a third feature value from an input signal for learning;
A learning feature value conversion step in which the signal processing device generates a fourth feature value obtained by performing the noise or acoustic distortion reduction process on the third feature value;
A Gaussian mixture model learning step in which the signal processing device acquires a Gaussian mixture distribution model by a maximum likelihood estimation method using the fourth feature amount as learning data;
A maximum likelihood Gaussian distribution calculating step in which the signal processing device obtains an index of a Gaussian distribution in the Gaussian mixture distribution model giving the maximum likelihood for each time, and obtains a time series of the index;
A storing step in which the signal processing apparatus stores the time series of the index in the storage unit as a parameter of the mixed distribution model;
The signal processing method according to claim 1 or 2, characterized in that it contained. A storage unit for storing a mixed distribution model in which a voice including noise or acoustic distortion or a clean voice is learned;
A feature quantity generator for generating a first feature quantity from an input signal;
A feature amount conversion unit that converts the first feature amount into a second feature amount subjected to noise or acoustic distortion reduction processing;
A speaker feature value generating unit for generating speaker feature values expressing speaker characteristics;
A connecting unit that generates a connected feature value obtained by connecting the second feature value and the speaker feature value;
Based on the parameters of the mixed distribution model stored in the storage unit, a posterior probability indicating the probability that the connected feature amount corresponds to each distribution of the mixed distribution model is calculated, and a clean speech feature having the highest posterior probability A matching unit for obtaining a quantity as a clean speech feature corresponding to the input signal;
An output unit that outputs an emphasized speech obtained by multiplying the input signal by a filter composed of clean speech feature values obtained by the matching unit;
A signal processing apparatus comprising: A signal processing program for causing a signal processing device to execute,
The signal processing apparatus includes a storage unit that stores a mixed distribution model in which a voice including noise or acoustic distortion or a clean voice is learned,
A feature value generation step for generating a first feature value from an input signal;
A feature amount conversion step of converting the first feature amount into a second feature amount subjected to noise or acoustic distortion reduction processing;
A speaker feature generating step for generating a speaker feature expressing a speaker feature;
A connecting step of generating a connected feature value obtained by connecting the second feature value and the speaker feature value;
Based on the parameters of the mixed distribution model stored in the storage unit, a posterior probability indicating the probability that the connected feature amount corresponds to each distribution of the mixed distribution model is calculated, and a clean speech feature having the highest posterior probability A collation step for obtaining a quantity as a clean speech feature corresponding to the input signal;
An output step of outputting an enhanced speech signal obtained by multiplying the input signal by a filter composed of clean speech feature values obtained in the collating step;
A signal processing program for causing the signal processing device to execute.

Images