JP2009139894A - Noise suppressing device, speech recognition device, noise suppressing method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a noise suppressing device capable of removing noise from speech in which a plurality of kinds of changing noises and unknown noises are superimposed. <P>SOLUTION: The noise suppressing device includes: a speech noise acoustic model storage section 14 for storing a plurality of speech noise acoustic models that is one classification of speech and noise acoustic model for a training; a synthesized acoustic model storage section 15 for storing a combined acoustic model that is the acoustic model where two or more classifications of speech and noise are synthesized; a dictionary information storage section 17 for storing dictionary information for relating a label of the classification of speech and noise to the acoustic model; a receiving section 19 for receiving a noise superimposed speech data; a label recognition section 20 for recognizing a label corresponding to the noise superimposed speech data, by using the acoustic model and the dictionary information; and a particle filter noise suppressing section 21 for generating a clean speech data, by sampling a particle from the acoustic model according to the recognized label, and by updating the sampled particle. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a noise suppression device that suppresses noise.

  Conventionally, many noise-resistant speech recognition techniques have been proposed for recognition of noise-superimposed speech. For stationary noise, there are Spectral Subtraction (for example, see Non-Patent Document 1) and Parallel Model Combination (PMC) (for example, see Non-Patent Document 2). A technique based on Minimum Mean-Squared Error (MMSE) using Gaussian Mixture Model (GMM) (for example, see Non-Patent Document 3) can cope with fluctuations in input speech. Furthermore, recently, research on non-stationary noise has become active (see, for example, Non-Patent Documents 4 and 5). These methods generally take into account only one type of noise and assume that the noise can be modeled with one model. However, in an actual environment, there are many not only stationary noises but also sudden noises. Another problem is how to estimate the actual noise from the input.

So far, the inventors of the present application have used a multi-model synthesis, obtain a maximum likelihood label sequence by multipath search, and perform noise suppression using a multi-noise model (MM-NS: Multi-Model Noise Suppression). Has been proposed (see, for example, Non-Patent Documents 6 and 7). Using a noise model obtained from learning data and a combined model obtained by combining them, a noise label sequence is obtained by noise label recognition by multipath search, and an MMSE noise suppression method using GMM based on the noise label sequence (for example, non-processing) Noise suppression is performed by a method in which Patent Document 3) is extended to a multiple synthesis model. By obtaining a noise superimposed speech model not by learning but by model synthesis, there is an advantage that a synthesized distribution of speech and noise at an SNR not included in the learning data can be created. Further, by the MMSE noise suppression method using GMM, each distribution is weighted by the posterior probability for the input speech, and noise is estimated. Therefore, noise suppression processing is performed with a noise model reflecting the input noise superimposed speech. However, since the performance can be improved by increasing the number of mixed distributions of each noise model, the granularity of each noise model is better as it is more detailed.
S. F. Boll, “Suppression of acoustic noise in speculation using subtraction”, IEEE Trans. Acoustic. Speech Signal Process. , Vol. 27, no. 27, p. 113-120, 1979 M.M. F. J. et al. Gales, “Model-based technologies for noise robust speech recognition”, PhD thesis, University of Cambridge, 1995. J. et al. C. Segura, A .; de la Torre, M.M. C. Benitez, A.M. M.M. Peinado, “Model-based compensation of the additive noise for continuous speech recognition. Explorations using the AURORAII database and tasks.” of EUROSPEECH 2001, vol. 1, p. 221-224, 2001 K. Yao, K .; K. Paliwal, S.M. Nakamura, “Noise adaptive speech recognition based on sequential noise parameter estimation”, Speech Communication, vol. 42, no. 1, p. 5-23, 2004 M.M. Fujimoto, S .; Nakamura, “A non-stationary noise suppression method based on particle filtering and Polyk averaging”, IEICE Trans. Inf. & Syst. , Vol. E89-D, no. 3,2006 Tsuji, Toriyama, Kogure, “Noise Suppression Method Based on Search Using Multiple Noise Synthesis Models”, IEICE Tech. SP2007-16, p. 49-54, 2007 Tsuji, Toriyama, Kogure, “Noise Suppression Based on Multipath Search Using Multiple Noise Synthesis Models”, Sound Lecture, p. 151-154, 2007

  However, the technique (MM-NS) that uses multiple model synthesis, obtains the maximum likelihood label sequence by multipath search, and performs noise suppression using multiple noise models is based on the noise model estimated from the learning data. There is a possibility that a close distribution to unknown noise that does not exist in the learning data may be used, but there is a problem that it is difficult to deal with strictly. That is, more proactive responses to fluctuations in input noise and unknown noise have been problems.

  The present invention has been made in order to solve this problem, such as a noise suppression device that can appropriately remove noise from a voice on which a plurality of types of noise are superimposed and can also remove fluctuating noise. The purpose is to provide.

  In order to achieve the above object, a noise suppression device according to the present invention is one type of speech data included in training data including training speech data and training noise data, or an acoustic model of one type of noise data. A speech noise acoustic model storage unit that stores a plurality of speech noise acoustic models, and a synthesized acoustic model that is an acoustic model in which two or more types of speech data and noise data included in the training data are synthesized is stored A dictionary that stores an acoustic model storage unit, dictionary information that is information that associates the speech noise model or the synthetic acoustic model with a label that is information identifying the type of speech and noise superimposed in the training data An information storage unit; a reception unit that receives noise-superimposed speech data that is speech data on which noise is superimposed; and the speech noise acoustic model. , Using the synthetic acoustic model and the dictionary information, a label recognition unit for recognizing a label corresponding to the noise-superimposed speech data for each frame, and sampling particles from the acoustic model corresponding to the label recognized by the label recognition unit And a particle filter noise suppression unit that generates clean audio data in which noise of the noise superimposed audio data is suppressed by updating the sampled particles.

  With such a configuration, it is possible to know what kind of noise is included in the noise-superimposed speech data by performing label recognition, so a new kind of suddenly entering the noise-superimposed speech data Appropriate noise suppression can be performed for noise. Further, since noise suppression is performed using a particle filter, appropriate noise suppression can be performed even when the noise fluctuates.

  In the noise suppression device according to the present invention, the particle filter noise suppression unit includes a feature amount extraction unit that extracts a feature amount for each frame of the noise-superimposed speech data, a feature amount corresponding to the noise-superimposed speech data, Using a speech noise acoustic model or a synthetic acoustic model corresponding to the label recognized by the label recognition unit, a plurality of particles are sampled from the acoustic model corresponding to the label recognized by the label recognition unit, and the sampled plurality of samples Using the particle filter means for updating the particles and calculating the weight of each updated particle, the particles updated by the particle filter means, and the weight calculated by the particle filter means, a noise component is calculated for each frame. Noise component calculating means for performing the noise superimposed voice data From said noise component calculating means removes noise components calculated, and noise suppression means for obtaining clean speech data may be provided.

  With such a configuration, it is possible to know what noise component is superimposed on the audio data using the label recognition result, and it is possible to sample particles from the acoustic model corresponding to the type of noise. The noise component can be removed from the noise superimposed voice data.

The noise suppression apparatus according to the present invention further includes a label language model storage unit that stores a label language model that is a language model of the label, and the label recognition unit includes the speech noise acoustic model, the synthesized acoustic model, The label corresponding to the noise-superimposed speech data may be recognized for each frame using the label language model and the dictionary information.
With such a configuration, the accuracy of label recognition can be improved by using a label language model.

  Further, in the noise suppression device according to the present invention, a training data storage unit that stores the training data, and a label storage unit that stores training label information that is information along a time series of labels corresponding to the training data, The speech noise acoustic model and the synthesized acoustic model are generated from the training data stored in the training data storage unit using the training label information, and the speech noise acoustic model storage unit and the synthesized acoustic model storage unit A label language model of a label is generated using a model generation unit that accumulates and training label information stored in the label storage unit, and is stored in the label language model storage unit and the dictionary information is generated. And a label language model generation unit that accumulates in the dictionary information storage unit.

  With such a configuration, it is possible to generate a speech noise acoustic model, a synthetic acoustic model, dictionary information, and a label language model for a label used for label recognition, and label recognition is performed using the generated model. Can do.

  The speech recognition apparatus according to the present invention includes the noise suppression device, an acoustic model storage unit for storing an acoustic model related to speech data to be speech-recognized, and dictionary information for speech recognition used in speech recognition. A dictionary information storage unit for speech recognition to be stored, a language model storage unit to store a language model related to a recognition target language for speech recognition, and clean speech data generated by the noise suppression device, the acoustic model, the speech recognition Dictionary information, a speech recognition unit that recognizes speech using the language model, and an output unit that outputs a speech recognition result by the speech recognition unit.

  With such a configuration, since speech recognition is performed using clean speech data in which noise is suppressed, speech recognition from received noise superimposed speech data can be accurately performed.

  According to the noise suppression device and the like according to the present invention, it is possible to effectively remove two or more types of noise from voices superimposed thereon, and to effectively remove fluctuating noise. . As a result, accuracy in processing such as speech recognition can be improved.

  Hereinafter, a noise suppression device and a speech recognition device according to the present invention will be described using embodiments. In the following embodiments, components and steps denoted by the same reference numerals are the same or equivalent, and repetitive description may be omitted.

(Embodiment 1)
A noise suppression apparatus according to Embodiment 1 of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram showing a configuration of a noise suppression apparatus 1 according to the present embodiment. The noise suppression apparatus 1 according to the present embodiment includes a training data storage unit 11, a label storage unit 12, a model generation unit 13, a speech noise acoustic model storage unit 14, a synthetic acoustic model storage unit 15, and a label language model. A generation unit 16, a dictionary information storage unit 17, a label language model storage unit 18, a reception unit 19, a label recognition unit 20, a particle filter noise suppression unit 21, and a storage unit 22 are provided.

  The training data storage unit 11 stores training data. Here, the training data includes voice data and noise data. Both the voice data and the noise data are for training used for learning the model. Voice data is data that is not noise, for example, voice data uttered by a human. The noise data is noise data such as a beep sound and machine noise. Using this training data, the model generation unit 13 described later learns the model.

  The process in which training data is memorize | stored in the training data storage part 11 is not ask | required. For example, training data may be stored in the training data storage unit 11 via a recording medium, and training data transmitted via a communication line or the like is stored in the training data storage unit 11. Alternatively, training data input via an input device such as a microphone may be stored in the training data storage unit 11. The storage in the training data storage unit 11 may be temporary storage in the RAM or the like of training data read from an external storage device or the like, or may be long-term storage. The training data storage unit 11 can be realized by a predetermined recording medium (for example, a semiconductor memory, a magnetic disk, an optical disk, etc.).

  The label storage unit 12 stores training label information, which is information along a time series of labels corresponding to training data. With this training label information, the type of voice data and noise data in the training data stored in the training data storage unit 11 is labeled. For example, the labels “beep”, “target”, and “beep.target” indicate that the training data includes a beep sound, a target utterance, and data in which the beep sound and the target utterance are superimposed. The target utterance is a desired utterance, that is, an utterance intended for processing, listening, and the like. For example, in the case of performing speech recognition, it is an utterance that is a target of speech recognition. Since this training label information is information of a label along a time series, for example, it includes information related to the time of training data (for example, a time code), and the period of training data corresponding to the label by the information. May be specified.

  The process in which training label information is memorize | stored in the label memory | storage part 12 is not ask | required. For example, training label information may be stored in the label storage unit 12 via a recording medium, and training label information transmitted via a communication line or the like is stored in the label storage unit 12. Alternatively, the training label information input via the input device may be stored in the label storage unit 12. The storage in the label storage unit 12 may be temporary storage in the RAM or the like of training label information read from an external storage device or the like, or may be long-term storage. The label storage unit 12 can be realized by a predetermined recording medium (for example, a semiconductor memory, a magnetic disk, an optical disk, etc.).

  The model generation unit 13 generates a speech noise acoustic model and a synthetic acoustic model from the training data stored in the training data storage unit 11 using training label information, and the speech noise acoustic model storage unit 14 and the synthetic acoustic model Each is stored in the storage unit 15. The speech noise acoustic model is an acoustic model of one kind of speech data or noise data included in training data. The type of audio data is, for example, “target utterance”, “other utterance”, or the like. The noise data is, for example, “beep sound”, “machine noise”, and the like. The acoustic model of voice data may be, for example, an acoustic model of voice data of a target utterance, an acoustic model of voice data of another person's utterance, or the like. The noise data acoustic model may be, for example, a beep acoustic model, a machine noise acoustic model, or the like. A synthetic acoustic model is an acoustic model in which two or more types of speech data and noise data included in training data are synthesized. “Two or more types of audio data and noise data” may be, for example, two or more types of audio data, one or more types of audio data and one or more types of noise data, Two or more types of noise data may be used. The combination of two or more types of speech data and noise data in the synthetic acoustic model may be all combinations of one type of speech data or noise data included in the training data, or some combinations. It may be. Even in the former case, it is preferable that the maximum number of combinations is determined. Moreover, in the latter case, for example, a combination included in training data may be included among a combination of one type of voice data or noise data, or a combination other than that. Further, the speech noise acoustic model and the synthesized acoustic model generated by the model generation unit 13 may or may not be adapted to the speaker.

  The model generation unit 13 uses the training label information to extract a desired voice data section, a desired noise data section, a section in which desired voice noise is superimposed, or the like from the training data. Can do. Therefore, for example, when the model generation unit 13 generates a beep sound noise acoustic model, a noise data section corresponding to the beep sound is extracted from the training information using the training label information, and the noise of the beep sound is extracted. A beep sound noise acoustic model is generated using the data. The model generation unit 13 generally performs modeling using a GMM (Gaussian Mixture Model), but may perform modeling using an HMM (Hidden Markov Model). In this embodiment, a case where modeling is performed by GMM will be described.

  A method in which the model generation unit 13 generates a speech noise acoustic model from training data is already known, and a description thereof will be omitted. Note that the number of mixture distributions may be changed for each type of audio data or noise data when modeling with the GMM. For example, the number of mixture distributions may be 200 in the case of target speech, and the number of mixture distributions may be 4 in the case of beep sounds.

  The model generation unit 13 may generate a synthesized acoustic model directly from the training data, or generate a speech noise acoustic model from the training data and synthesize two or more generated speech noise acoustic models. A synthetic acoustic model may be generated. In the present embodiment, it is assumed that the model generation unit 13 generates a speech acoustic model and generates a composite acoustic model by combining two or more speech noise acoustic models. For example, when two speech noise acoustic models are synthesized, the number of mixed distributions of the synthesized acoustic models is a number obtained by multiplying the number of mixture distributions of the two speech noise acoustic models. For example, the number of mixed distributions of a synthesized acoustic model obtained by synthesizing a speech noise acoustic model having a mixture distribution number “3” and a speech noise acoustic model having a mixture distribution number “2” is 2 × 3 = 6. As a model synthesis method, for example, a method of model synthesis similar to PMC (Parallel Model Combination) on model parameters is known, and detailed description thereof is omitted. The method of model synthesis using PMC is described in Non-Patent Document 2, for example.

Here, the merit of generating a synthesized acoustic model by synthesizing a speech noise acoustic model will be briefly described.

First, since the training data is finite, there may be a small number of samples in the section on which the corresponding noise and speech necessary for the synthetic acoustic model are superimposed. Therefore, rather than generating a synthetic acoustic model directly from training data, it is more accurate to generate a speech noise acoustic model from training data and generate a synthetic acoustic model by synthesizing two or more speech noise acoustic models. It is considered high. In addition, when generating a synthesized acoustic model by synthesizing a speech noise acoustic model, it is included in the training data by learning each individual model and generating a synthesized acoustic model with a combination of the respective mixed distributions. It can also include patterns of noise and voice combinations that are not. Therefore, the range that can be expressed by the model can be made wider than the model obtained directly from learning. In general, it is considered faster to generate a synthesized acoustic model by synthesizing an acoustic model than to generate an acoustic model from data. Therefore, it is considered that it is more appropriate to generate a synthesized acoustic model by synthesizing two or more speech noise acoustic models from the viewpoint of shortening the time for generating a synthesized acoustic model. Furthermore, in training data, it is difficult to appropriately specify a section where speech and noise overlap or a section where two or more types of noise overlap. Therefore, it is considered that a more accurate model is obtained by specifying more specific simple speech-only sections or noise-only sections and synthesizing the acoustic models corresponding to these sections.

  Further, as will be described later, the model generation unit 13 may receive audio data of background noise included in the noise superimposed audio data and generate an acoustic model corresponding to the background noise.

  The speech noise acoustic model storage unit 14 stores a plurality of speech noise acoustic models that are acoustic models of one type of speech data or noise data included in training data including training speech data and noise data. The audio noise acoustic model storage unit 14 can be realized by a predetermined recording medium (for example, a semiconductor memory, a magnetic disk, an optical disk, or the like).

  The synthetic acoustic model storage unit 15 stores a synthetic acoustic model, which is an acoustic model obtained by synthesizing two or more types of data among speech data and noise data included in training data. The synthetic acoustic model storage unit 15 can be realized by a predetermined recording medium (for example, a semiconductor memory, a magnetic disk, an optical disk, etc.).

  The label language model generation unit 16 generates a label language model of the label using the training label information stored in the label storage unit 12, accumulates the label language model in the label language model storage unit 18, and generates dictionary information. It accumulates in the dictionary information storage unit 17. Here, the label language model may be, for example, an N-gram model of a label or a grammar. The grammar may be, for example, a network grammar, a CFG (Context Free Grammar), or a grammar that uses a probability in those grammars. The N-gram model and grammar are known in natural language processing and speech recognition, and detailed description thereof is omitted. In the present embodiment, a case where the label language model is an N-gram model will be described. The dictionary information is information that associates a label with a speech noise acoustic model or a synthetic acoustic model. The dictionary information may be information having information for identifying a label and information for identifying a speech noise acoustic model or information for identifying a synthetic acoustic model, for example. The information for identifying the label may be the label itself, and the information for identifying the speech noise acoustic model or the information for identifying the synthetic acoustic model may be the name of the model. Accordingly, the dictionary information may be, for example, information that associates “beep” that is information for identifying a beep sound label with “beeep” that is information for identifying a speech noise acoustic model corresponding to the beep sound. . The label language model generation unit 16 may refer to a speech noise acoustic model, a synthetic acoustic model, or the like in order to acquire information for identifying the model when generating the label language model and dictionary information. .

  Note that the method of generating the label language model by the label language model generation unit 16 is the same as the method of generating an ordinary N-gram language model and the method of generating a grammar (this time, the word of the language model etc. The explanation is omitted. In addition, when the label language model generation unit 16 generates an N-gram model, the N value of the N-gram model of the generated label is determined in advance. N may be, for example, 2 (bigram), 3 (trigram), or both.

  The dictionary information storage unit 17 stores dictionary information, which is information for associating a speech noise acoustic model or a synthetic acoustic model with a label that is information identifying the type of speech and noise superimposed in the training data. The dictionary information storage unit 17 can be realized by a predetermined recording medium (for example, a semiconductor memory, a magnetic disk, an optical disk, etc.).

  The label language model storage unit 18 stores a label language model of a label that is information for identifying the type of speech and noise superimposed in the training data. The label language model storage unit 18 can be realized by a predetermined recording medium (for example, a semiconductor memory, a magnetic disk, an optical disk, etc.). When the label language model includes a grammar, a part or all of the grammar may not be generated by the label language model generation unit 16, for example, it may be generated manually. Good. In that case, part or all of the label language model may be input from the outside and stored in the label language model storage unit 18.

  The accepting unit 19 accepts noise superimposed speech data that is speech data on which noise is superimposed. For example, the reception unit 19 may receive noise superimposed voice data input from an input device (for example, a microphone) or may receive noise superimposed voice data transmitted via a wired or wireless communication line. In addition, noise superimposed audio data read from a predetermined recording medium (for example, an optical disk, a magnetic disk, a semiconductor memory, etc.) may be received. The reception unit 19 may or may not include a device (for example, a modem or a network card) for reception. The reception unit 19 may be realized by hardware, or may be realized by software such as a driver that drives a predetermined device. There may be a recording medium (not shown) that temporarily stores the noise-superimposed audio data received by the receiving unit 19.

  The label recognition unit 20 includes a speech noise acoustic model stored in the speech noise acoustic model storage unit 14, a synthetic acoustic model stored in the synthetic acoustic model storage unit 15, and dictionary information stored in the dictionary information storage unit 17. Using the label language model stored in the label language model storage unit 18, the label corresponding to the noise superimposed speech data received by the receiving unit 19 is recognized. This recognition is performed in the same manner as the speech recognition except that words in speech recognition become labels and a model for each phoneme becomes a model for speech data, noise data, or a mixture thereof. Therefore, as the label recognition unit 20, components that perform conventional speech recognition processing using an acoustic model, a language model, and a dictionary can be used, and detailed description thereof is omitted.

  As a result of this label recognition, labels corresponding to the noise-superimposed speech data are specified along the time series. For example, the label “beep” is associated with 1 to 50 frames of the noise-superimposed audio data, and the label “beep.target” is associated with 51 to 200 frames. Information indicating the recognized label is stored in a recording medium (not shown).

  Note that the label recognizing unit 20 may acquire the maximum likelihood label sequence by performing a beam search or a multipath search for switching a plurality of models. For example, a search using a sound noise acoustic model or a synthetic acoustic model and a bigram may be performed in the first pass, and re-scoring using a trigram may be performed in the second pass.

  The particle filter noise suppression unit 21 samples clean particles from the acoustic model corresponding to the label recognized by the label recognition unit 20, and updates the sampled particles, whereby clean audio data in which noise in the noise superimposed audio data is suppressed is obtained. Is generated. Note that the particle filter noise suppression unit 21 may perform noise suppression in the feature amount space, or may perform noise suppression on the audio signal. In the former case, the output of the particle filter noise suppression unit 21 is a feature quantity from which the noise component has been removed. In the latter case, the output of the particle filter noise suppression unit 21 is an audio signal from which the noise component has been removed. Signal.

  FIG. 2 is a block diagram showing a detailed configuration of the particle filter noise suppression unit 21 according to the present embodiment. The particle filter noise suppression unit 21 includes a feature amount extraction unit 31, a particle filter unit 32, a noise component calculation unit 33, and a noise suppression unit 34.

  The feature amount extraction unit 31 extracts a feature amount for each frame of the noise superimposed speech data. For example, the feature amount extraction unit 31 may generate a log mel spectrum corresponding to the noise superimposed voice data by performing a mel filter bank analysis on the noise superimposed voice data for each frame. Further, the feature quantity extraction unit 31 may extract other feature quantities.

  The particle filter unit 32 uses the feature amount corresponding to the noise-superimposed speech data and the speech noise acoustic model or the synthesized acoustic model corresponding to the label recognized by the label recognition unit 20 to generate a label recognized by the label recognition unit 20. A plurality of particles are sampled from the corresponding acoustic model, the plurality of sampled particles are updated, and the weight of each updated particle is calculated. A specific expression of this processing will be described later.

  The noise component calculation unit 33 calculates a noise component for each frame using the particles updated by the particle filter unit 32 and the weight calculated by the particle filter unit 32. A specific expression of this processing will be described later.

    The noise suppression means 34 removes the noise component calculated by the noise component calculation means 33 from the noise superimposed voice data, and acquires clean voice data. This noise component removal may be performed, for example, by forming a Wiener filter from the estimated noise component and obtaining a speech waveform in the time domain by noise suppression by filtering, or a noise component in the log mel spectral domain Subtraction may be performed. In the former case, a Wiener filter in the logarithmic mel spectrum region can be configured, and by performing the filtering process, a speech waveform can be estimated from the input noise superimposed speech data. On the other hand, in the latter case, the noise component in the log mel spectrum region is removed, and the clean speech data from which the noise component has been removed becomes a log mel spectrum. In the present embodiment, the former case will be described.

  The accumulating unit 22 accumulates the clean voice data whose noise is suppressed by the particle filter noise suppressing unit 21 in a predetermined recording medium. The recording medium is, for example, a semiconductor memory, an optical disk, a magnetic disk, or the like, and may be included in the storage unit 22 or may exist outside the storage unit 22. Further, this recording medium may or may not store clean audio data temporarily.

  In the present embodiment, the case where clean speech data after noise suppression is stored in the noise suppression device 1 will be described. However, the noise suppression device 1 replaces the storage unit 22 with clean speech after noise suppression. You may provide the output part which outputs data. The output may be, for example, transmission via a communication line to a predetermined device, audio output by a speaker, or accumulation in a recording medium. Note that the output unit may or may not include an output device (for example, a speaker or a communication device). The output unit may be realized by hardware, or may be realized by software such as a driver that drives these devices.

  The training data storage unit 11, the label storage unit 12, the speech noise acoustic model storage unit 14, the synthetic acoustic model storage unit 15, the dictionary information storage unit 17, the label language model storage unit 18, and the storage unit 22 store clean speech data. Of the recording media not shown, any two or more recording media may be realized by the same recording medium or may be realized by separate recording media. In the former case, for example, an area storing training data is the training data storage unit 11, and an area storing training label information is the label storage unit 12.

Next, a noise suppression method used in the noise suppression apparatus 1 according to the present embodiment will be described.
In the log mel spectrum region, the feature vector corresponding to the noise-superimposed speech data in the t-th frame is x _t , the feature vector corresponding to the clean speech data is s _t , and the nth of the N types of noise data. Let n _t (n) be a feature vector corresponding to noise data. Note that x _t , s _t , and n _t (n) may be simply referred to as noise superimposed speech, clean speech, and noise. These state space models are defined. The observation model is expressed as follows.

Here, n _t is a synthesized noise including all noises in the t-th frame. g (s _t , n _t ) is a mismatch component between the clean speech s _t and the noise superimposed speech x _t . v _t represents observation noise, and Σ _x is a covariance matrix of x _t .

  In the noise suppression apparatus 1 according to the present embodiment, since label recognition is performed for each frame, it is possible to distinguish between a speech section (a section including a target utterance) and a section that is not. Therefore, noise suppression can be performed separately for each section. The mismatch component for each mixture distribution can be defined as follows.

Here, μ _{x, l} is the l-th average vector in the mixture distribution of the acoustic model corresponding to the noise superimposed speech data, and the mixture distribution of the acoustic model corresponding to the clean speech data for the target speech section. a second l-th mean vector mu _{s, l} in, is produced in the synthesis of the noise component in the acoustic model corresponding to the estimated noise data. Which noise component is the synthesis can be known from the recognized label. Note that only noise components are synthesized for μ _{x, l} in other sections. Ε is a small positive value for adjusting the residual signal power after noise suppression.

Further, it is assumed that the system model of the l-th mixture distribution n _{t, l} (n) of the acoustic model corresponding to the noise n _t (n) can be modeled as follows with a random walk assumption.

Here, u _t (n, l) is system noise, and Σ _{ut (n, l)} = Σ _{n (n), l} is a covariance matrix of noise n _{t, l} (n).

  The state space model is expressed by equations (2) and (5), and a particle filter is defined in the same manner as in Non-Patent Document 5 described above, and is integrated with noise suppression in the noise suppression apparatus 1 according to the present embodiment.

  Similarly, the noise suppression apparatus 1 according to the present embodiment uses an extended Kalman particle filter, resampling (residual sampling), and a Markov chain Monte Carlo (MCMC). The noise suppression apparatus 1 uses a mixture distribution of acoustic models corresponding to recognized labels as a prior distribution of each particle.

  FIG. 3 is a diagram for explaining an outline of particle filtering in the noise suppression apparatus 1 according to the present embodiment. First, the particles for the noise distribution are sampled from an acoustic model of background noise. This acoustic model is generated from, for example, data of a section with only background noise. At this stage, it is the same as Non-Patent Document 5 described above. Next, a frame in which a new noise “noise 1” is detected at time t by label recognition is created from learning data in advance and stored in the speech noise acoustic model storage unit 14 or the synthesized acoustic model storage unit 15. New particles are sampled from the acoustic model of “Noise 1”. In the next frame, since “Noise 1” is continuously detected, it is considered that noise having a similar distribution exists, and it can be easily estimated from “Noise 1” particles at time t. . Therefore, the “noise 1” particle at time t + 1 is estimated using an extended Kalman filter, as in the conventional method. When “noise 1” is no longer detected at time t + 2, the particle is also removed from the particles used for estimation. Thus, the noise suppression apparatus 1 according to the present embodiment can handle sudden noise by sampling particles from the acoustic model corresponding to the recognized label. In addition, when performing particle filtering in each frame, in order to estimate a noise closer to the noise-superimposed speech, a combination of particles between clean speech and noise is considered and used for estimation to improve estimation accuracy. .

Next, the particle filter will be described.
Here, a particle filter called “Sequential importance sampling” is defined. When the state space model is Equations (2) and (5), the posterior probability density function is as follows.

Here, n _{0: t} = {n ₀ ,..., N _t } and x _{0: t} = {x ₀ ,..., X _t }. In the particle filter, this posterior probability density function is approximated as follows.

Here, (k) is the particle number, and K is the total number of particles. δ (·) is a Dirac delta function, and w _t ^(k) represents the weight for the kth particle at time t. Since sampling from p (n _{0: t} | x _{0: t} ) itself is difficult, q (n _{0: t} | x _{0: t} ) called impedance density is introduced. Then, the weight of each particle can be expressed as follows.

Here, by assuming that p ( _nt ^(k) | _nt-1 ^(k) ) = q ( _nt ^(k) | n0 _{: t} ^(k) , x0 _{: t} ), (9) The equation can be simplified as:

  In the particle filter, sampling is performed from an acoustic model of clean speech and an acoustic model of noise. As for the acoustic model of noise, when noise is detected during label recognition, using the acoustic model of background noise estimated using data for a short time from the start of the received noise superimposed speech data in the initial frame, Uses an acoustic model of noise corresponding to the recognized label. Sampling from each model is defined as follows:

[Sampling from acoustic model of clean speech]

Here, μ _{s, l} , Σ _{s, l} are the l-th average vector and covariance matrix in the mixture distribution of the acoustic model corresponding to the clean speech data, respectively. L _s is the number of mixtures in the mixture distribution.

[Sampling from acoustic model of noise]

Here, μ _{n (n), l} , Σ _{n (n), l} are noises corresponding to the labels recognized in the frame (may be a single noise or a synthesized noise). It is the l-th average vector and covariance matrix in the mixture distribution of the acoustic model corresponding to the data. L _{n (n)} is the number of mixtures in the mixture distribution.

A pair of noise-superimposed speech particles is created by synthesizing the speech particles sampled in this way and the noise particles. Next, using an extended Kalman filter, and update each particle _to estimate the average of ^{n t (k)} _^ and ^{n t (k),} and a dispersion ^ Σ _nt ^(k). Here, ^ A indicates that a top (A) is added to A.

Here, t | t−1 indicates a parameter of the t-th frame predicted from the (t−1) -th frame.

  After particle filtering, similar to Non-Patent Document 5 described above, residual resampling and MCMC (Metropolis-Hasting sampling) are performed.

  Also, by using such a method, the current noise distribution can be estimated, and a single distribution indicating the noise distribution can be obtained. In the conventional noise suppression method that performs label recognition, the acoustic model of the noise was also shown as a mixture distribution, so it was necessary to calculate the mixture distribution. Will be reduced. For example, a specific case where the number of mixed acoustic models of clean speech is 200 will be considered. When the noise acoustic model is represented by a mixture distribution with a mixture number of 4, the mixture number of the acoustic model of the noise superimposed speech is 200 × 4 = 800, and calculation is performed for all of the 800 mixture numbers. Become. On the other hand, if the noise acoustic model has a single distribution (mixing number = 1), the number of mixing of the acoustic model of the noise-superimposed speech is 200 × 1 = 200. It can be seen that the calculation amount is reduced.

  In the above description, an extended Kalman filter (EKF: Extended Kalman Filter) is used in the previous stage of the particle filter, but an Unsented Kalman filter (UKF: Unscented Kalman Filter) may be used instead. The EKF is a nonlinear model derived by a first-order approximation of Taylor expansion, and as an approximation method with higher accuracy, UKF using U transformation (Unsented Transformation) has been proposed. By using it, it is possible to approximate up to a quadratic term. By simply replacing EKF with UKF, it is possible to introduce UKF into the noise suppression method according to the present embodiment.

Please refer to the following literature for UKF.
Literature: S.M. J. et al. Julier, J. et al. K. Uhlmann, “A new extension of the Kalman filter to nonlinear systems”, Proc. AeroSense: Int. Symp. Aerospace / Defense Sensing, Simul. and Controls, Orlando, FL, 1997

  Literature: Takashi Tanibe, “Kalman filter and adaptive signal processing”, Corona, 2005

  Literature: Masakita Yamakita, “What is UKF (Unscented Kalman Filter)?”, System / Control / Information, vol. 50, no. 7, p. 261-266, 2006

For the particle filter using UKF, refer to the following document.
Literature: R.D. van der Merwe, A.M. Doucet, N.M. de Freitas, E .; Wan, “The unscented particulate filter”, Technical Report CUED / FINFENG / TR 380, Cambridge University Engineering Department, 2000

  Next, a noise suppression method based on a particle filter will be described. That is, the procedure until noise suppression based on GMM is described using the above definition.

(I) Initialization (for frame number t = 0)
Each particle n ₀ ⁽ⁱ⁾ (i = 0,..., I) is sampled from an acoustic model of background noise.

(II) For each frame (t = 1, 2,..., T) (II-1) Importance sampling step (particle filter):

If the recognized label indicates that the target speech is included in the frame, the speech particle s _t ⁽ⁱ⁾ is sampled from the acoustic model of the clean speech according to the equation (11).
When a new noise n (n) is detected in the frame by the recognized label, noise particles are sampled by the equation (12).
A set of noise superimposed speech particles is created by combining s _t ⁽ⁱ⁾ and n _t-1 ^(j) . Here, the total number of particles is K = I × (J + 1).

For k = 1,..., K, each particle is updated by the extended Kalman filter. That is, (13) and _(14), estimates that ^{n t} mean ^(k) _^ ^{n t (k),} and a dispersion ^ sigma _nt ^(k).

k = 1, ..., K for the weight _{^{w t (k) αw t-}} 1 (k) p | to calculate the ^{_{(x t n t (k)}} ). For w _t−1 ^(k) , a value calculated one frame before is used. ^{_{Further, p (x t | n t}} (k)) is the estimated average _^ ^{n t (k),} it can be calculated using the dispersion ^ Σ _nt ^(k).

For k = 1,..., K, the calculated w _t ^(k) is normalized to calculate a normalization weight ^ w _t ^(k) . Normalized weights _^ ^{w t (k)} _can be calculated by dividing by ^{w t} a ^(k), it was summed for ^k _w ^{t (k).}

(II-2) Residual sampling step:
Proliferate / suppress particles with high / low importance weight. This method of resampling is already known and will not be described in detail.

(II-3) MCMC step:
Apply Metropolis-Hastings sampling. This method is already known and will not be described in detail.

(II-4) Estimation of noise posterior distribution:
The posterior distribution is estimated from the particles as follows.
Here, μ _{^ nt} and Σ _{^ nt} are an estimated average vector of a noise acoustic model and a covariance matrix, respectively.

(II-5) Noise suppression using GMM-based labels for clean speech:
The clean speech is estimated as follows using the mismatch component.

Here, μ _{x, l} , Σx _{, l} are the average vector and covariance matrix of the acoustic model corresponding to the label of each frame, respectively. These are estimated from the acoustic model N (μ _{^ nt} , Σ _{^ nt} ) of noise that is estimated to be clean speech as appropriate by first-order Taylor series expansion (see next document).

  Literature: P.M. J. et al. Moreno, “Speech recognition in noisy environments”, PhD thesis, Carnegie Mellon University, Pittsburgh, Pennsylvania, 1996.

Next, the operation of the noise suppression apparatus 1 according to the present embodiment will be described using the flowchart of FIG.
(Step S101) The model generation unit 13 determines whether it is time to generate a model. If it is time to generate a model, the process proceeds to step S102; otherwise, the process proceeds to step S106. For example, when the noise suppression apparatus 1 receives an instruction to generate a model, the model generation unit 13 may determine that it is time to generate a model, and training data is accumulated in the training data storage unit 11. When training label information is accumulated in the label storage unit 12, it may be determined that it is time to generate a model, or it may be determined that it is time to generate a model at other timing.

  (Step S102) The model generation unit 13 uses the training label information stored in the label storage unit 12 to identify one type of speech data or one type of noise data, and Corresponding speech noise acoustic models are respectively generated and stored in the speech noise acoustic model storage unit 14. At the time of accumulation, a label (for example, “beep”) indicated by the training label information may be accumulated in association with the speech noise acoustic model.

  (Step S103) The model generation unit 13 reads out two or more speech noise acoustic models stored in the speech noise acoustic model storage unit 14, generates a synthesized acoustic model by synthesizing them, and a synthesized acoustic model storage unit Accumulate in 15

  Further, as described above, instead of covering all combinations of the speech noise acoustic model, a synthetic acoustic model corresponding to the combination indicated by the training label information stored in the label storage unit 12 is generated. Also good. In addition, when the synthetic acoustic model is accumulated, a label (for example, “beep.target” or the like) indicated by the training label information may be accumulated in association with the synthetic acoustic model.

  (Step S <b> 104) The label language model generation unit 16 generates a label language model using the training label information stored in the label storage unit 12 and stores the label language model in the label language model storage unit 18.

  (Step S <b> 105) The label language model generation unit 16 generates dictionary information and stores it in the dictionary information storage unit 17. Then, the process returns to step S101. As described above, this dictionary information associates a label with a speech noise acoustic model or a synthesized acoustic model. Therefore, for example, the dictionary information may be information that associates label information associated with the acoustic model with a label name. In this case, for example, the label information “beep” associated with the acoustic model and the label name “beep” are associated in the dictionary information.

  (Step S106) The reception unit 19 determines whether noise-superimposed voice data has been received. If accepted, the process proceeds to step S107, and if not, the process returns to step S101.

  (Step S107) The label recognition unit 20 recognizes a label corresponding to the noise-superimposed speech data using the speech noise acoustic model, the synthesized acoustic model, the label language model, and dictionary information. Then, the recognition result is stored in a recording medium (not shown).

  (Step S108) The particle filter noise suppression unit 21 uses the recognized label to generate clean voice data in which noise of the noise superimposed voice data is suppressed. Details of this processing will be described later with reference to the flowchart of FIG.

(Step S109) The storage unit 22 stores clean sound data after noise suppression in a recording medium (not shown). Then, the process returns to step S101.
In the flowchart of FIG. 4, the process ends when the power is turned off or the process ends.

FIG. 5 is a flowchart showing details of the noise suppression (step S108) processing in the flowchart of FIG.
(Step S <b> 201) The model generation unit 13 first generates a background noise acoustic model from the received noise-superimposed speech data, and stores the background noise acoustic model in the speech noise acoustic model storage unit 14. The particle filter noise suppression unit 21 performs sampling from an acoustic model of the background noise. Background noise is noise included in the background when the label recognition unit 20 does not recognize any label. Therefore, an acoustic model is generated using a predetermined number of frames of noise superimposed speech data that does not include beep sound, machine noise, target speech, etc., for example, noise superimposed speech data at the initial stage of input, and the above-mentioned acoustic model is generated from the acoustic model. Background noise sampling is performed as described in “(I) Initialization”. Note that the model generation unit 13 may appropriately generate a synthesized acoustic model obtained by synthesizing the acoustic model of the background noise with another acoustic model or a synthesized acoustic model.

(Step S202) The particle filter noise suppression unit 21 sets a counter t to 1.
(Step S203) The particle filter noise suppression unit 21 acquires a label corresponding to the frame t of the noise-superimposed speech data. This label is a label recognized by the label recognition unit 20 and is used in the particle filter means 32 and the noise component calculation means 33.

(Step S204) The feature amount extraction unit 31 extracts the feature amount of the frame t of the noise superimposed speech data. In the present embodiment, the feature amount extraction unit 31 generates a log mel spectrum by performing a mel filter bank analysis on the frame t of the noise superimposed speech data.
(Step S205) The particle filter means 32 performs a particle filter process on the feature quantity of the frame t.

  Specifically, when the label corresponding to the acquired frame t indicates that the target utterance is included, the sound particles are sampled from the sound model of the sound data as shown in Equation (11). Further, when the label corresponding to the acquired frame t indicates that noise is included, noise particles are sampled from the acoustic model corresponding to the noise as shown in Equation (12). When both the audio particles and the noise particles are sampled, the particle filter unit 32 generates a set of noise superimposed audio particles obtained by synthesizing both. In this case, the total number K of particles is K = I × J + I. Here, I is the number of audio particles, and J is the number of noise particles. Also, “I” in the second term on the right side is the number of background noise particles.

Moreover, the particle filter means 32 performs update using an extended Kalman filter for all particles. That is, the particle filter unit 32, for each particle _to estimate the average of ^{n t (k)} _^ and ^{n t (k),} and a dispersion ^ Σ _nt ^(k).

Further, the particle filter unit 32, for all particles, the weight _{^{w t (k) αw t-}} 1 (k) p | calculating the ^{_{(x t n t (k)}} ). For w _t−1 ^(k) , a value calculated one frame before is used. ^{_{Further, p (x t | n t}} (k)) is the estimated average _^ ^{n t (k),} it can be calculated using the dispersion ^ Σ _nt ^(k). Further, the particle filter means 32 normalizes the calculated weight w _t ^(k) .

  Thereafter, the particle filter unit 32 applies Residual sampling or Metropolis-Hastings sampling.

  In this particle filter processing, when the label acquired in step S203 matches the previous frame, that is, the label of the frame (t−1), the particle filter means 32 determines whether the sound particle or It is not necessary to sample noise particles. That is, in this case, the particle filter unit 32 performs the particle filtering process using the estimation result in the previous frame. On the other hand, if the label acquired in step S203 does not match the label of the previous frame, the particle filter means 32 discards the particles so far, and the expression (11) or ( As shown in equation (12), sound particles and noise particles are appropriately sampled, and particle sampling processing is performed.

(Step S206) The noise component calculating unit 33 calculates the mean ^ n _t ^{(k) of} each particle estimated by the particle filter unit 32, the variance ^ Σ _nt ^(k) , the weight of each particle, and clean audio data. A noise component corresponding to the frame t is calculated using a speech noise acoustic model corresponding to the frame t.

Specifically, the noise component calculation means 33 estimates the posterior distribution of noise using equation (15). Particle filter means 32 has been calculated, normalized weights _^ ^{w t (k)} and the mean _^ ^{n t (k),} by using a dispersion ^ sigma _nt ^(k), the noise component calculating means 33, (15) The central expression of can be obtained.

Next, the noise component calculation unit 33 calculates Expression (17). In Expression (17), w _{s and l} are obtained from the acoustic model corresponding to the sound data. Further, μ _{x, l} , Σx _{, l} are the average vector and covariance matrix of the acoustic model corresponding to the label of each frame, respectively. Since the noise is calculated by the equation (15), if the label recognized by the label recognition unit 20 indicates that the target utterance is not included, μ _{x, l} , Σ _{x, l} are , (15) is equal to the noise average vector and covariance matrix. On the other hand, when the label recognized by the label recognition unit 20 indicates that the target utterance is included, the noise model calculated by the equation (15) and the target utterance stored in the speech noise acoustic model storage unit 14 are used. The average vector and covariance matrix of the synthesis model with the acoustic model corresponding to is calculated. The calculated average vector or the like becomes μ _{x, l} , Σ _{x, l} .

Further, the noise component calculation means 33 calculates g (s _{t, l} , n _{t, l} ) using the equation (4) corresponding to the label recognized by the label recognition unit 20. (4) μ _{x, l} on the right side of the equation can be calculated as described above. Also, μ _{s, l on the} right side of the equation (4) can be calculated using acoustic data corresponding to the target utterance. In this way, the noise component calculation means 33 can calculate the second term on the right side of the equation (16).

  In FIG. 2, it is described that a speech noise acoustic model is input to the noise component calculation means 33. Strictly speaking, the noise component calculation means 33 uses an acoustic model corresponding to the target utterance. Only.

  In the present embodiment, since the case where the noise component is removed from the audio signal will be described, the noise component calculation means 33 uses the second term on the right side of the calculated equation (16) as the impulse response (time domain parameter). Convert. The noise component calculation unit 33 may use, for example, MEL-warped IDCT that performs conversion by mapping a mel spectrum to a linear spectrum (see, for example, the following document).

  Literature: W.M. Herbordt, T .; Moriuchi, M .; Fujimoto, T .; Jitsuhiro, S.M. Nakamura, “Hands-free speech recognition and communication on PDAs using microphone array technology”, Proc. of ASRU2005, pp. 302-307, 2005

  (Step S207) The noise suppression means 34 removes the noise component calculated by the noise component calculation means 33 from the noise superimposed voice data, thereby obtaining clean voice data. For example, when a noise component is given as an impulse response, the noise suppression unit 34 can obtain a frame t of clean voice data by convolving the impulse response with the frame t of noise superimposed voice data. The clean audio data frame t may be delivered to the storage unit 22 or may be temporarily stored in a recording medium (not shown) until the storage unit 22 passes the frame t.

(Step S208) The particle filter noise suppression unit 21 increments the counter t by 1.
(Step S209) The particle filter noise suppression unit 21 determines whether or not the frame t exists in the noise superimposed voice data. And when it exists, it returns to step S203, and when that is not right, it returns to the flowchart of FIG.

  Note that the flowchart of FIG. 4 shows a case where clean speech data is accumulated after noise suppression, but when noise suppression processing is performed as shown in the flowchart of FIG. Alternatively, the storage unit 22 may sequentially store the frames of the clean audio data.

Next, the operation of the noise suppression device 1 according to the present embodiment will be described using a specific example.
FIG. 6 is a diagram illustrating an example of training label information stored in the label storage unit 12. This training label information is information having a label and a time of training data corresponding to the label in association with each other. Time includes the beginning and the end. The unit in FIG. 6 is second. With this training label information, for example, the training data is a beep sound from 0.5 to 0.8 seconds, and the beep sound and the target utterance are superimposed from 0.8 to 1.0 seconds of the training data. It is shown that.

  In a situation where training data is stored in the training data storage unit 11 and training label information is stored in the label storage unit 12, the user operates an input device (for example, a mouse, a keyboard, etc.) not shown, Assume that an instruction to generate a model is input to the noise suppression apparatus 1. Then, the model generation unit 13 determines that it is time to generate a model (step S101). And the model production | generation part 13 specifies the label corresponding to one type of audio | voice data or one type of noise data, referring the label memory | storage part 12. FIG. Then, by acquiring the time corresponding to the specified label from the training label information, a section of one type of audio data or a section of one type of noise data is specified. Thereafter, the model generation unit 13 generates a speech noise / acoustic model corresponding to the identified section, and accumulates it in the speech noise / acoustic model storage unit 14 (step S102). In this accumulation, the model generation unit 13 accumulates the speech noise acoustic model in association with the name of the label corresponding to the speech noise acoustic model.

  Further, the model generation unit 13 specifies labels corresponding to two or more types of audio data and noise data while referring to the label storage unit 12. Then, audio data and noise data included in the specified label are specified, and audio noise acoustic models corresponding to the specified audio data and noise data are read from the audio noise acoustic model storage unit 14. Thereafter, the model generation unit 13 generates a synthesized acoustic model corresponding to the identified label by synthesizing the read voice noise acoustic models, and stores the synthesized acoustic model in the synthesized acoustic model storage unit 15 (step S103). . In this accumulation, the model generation unit 13 accumulates the synthesized acoustic model in association with the name of the label corresponding to the synthesized acoustic model.

  The label language model generation unit 16 generates an N-gram model of the label using the training label information stored in the label storage unit 12, and accumulates it in the label language model storage unit 18 (step S104). The label language model generation unit 16 also generates dictionary information using the training label information and stores it in the dictionary information storage unit 17 (step S105). This dictionary information is as shown in FIG. In FIG. 7, dictionary information is information having a label name, which is information for identifying a label, and information for identifying an acoustic model in association with each other. Note that in this specific example, the name of the label is used as information for identifying the acoustic model, so both are the same information.

  Next, the noise-superimposed sound collected by the microphone is converted into a digital signal by an A / D converter (not shown) and accumulated, and the accumulated series of noise-superimposed sound data is accepted by the accepting unit 19. (Step S106). Then, the label recognizing unit 20 performs label recognition by a method similar to speech recognition using a speech noise acoustic model, a synthesized acoustic model, dictionary information, and a label language model, that is, an N-gram model (step S107). For example, as shown in FIG. 8, when noise superimposed voice data on which a beep, machine noise, target utterance, or the like is superimposed is received, “beep” is received. And “beep.machine”, “machine”, and other labels are attached to each section of the noise-superimposed speech data. In addition, as shown in FIG. 9, the label recognition comprises recognition label information which is information having the recognized label name and the time of the label in association with each other, and the recognition label information is not shown. May be temporarily stored. The unit of time in the recognition label information shown in FIG. 9 is a frame number.

  Next, processing for suppressing noise by the particle filter noise suppressing unit 21 will be described. First, the model generation unit 13 generates an acoustic model of background noise from the noise-superimposed speech data in the section before the leftmost beep sound shown in FIG. 8 and stores it in the speech noise acoustic model storage unit 14. . In addition, the particle filter noise suppression unit 21 performs sampling from the generated acoustic model. Next, the noise component calculation unit 33 acquires the label name “beep” corresponding to the frame 1 from the recognition label information generated by the label recognition unit 20 (steps S202 and S203). Also, the feature quantity extraction unit 31 generates a log mel spectrum corresponding to the frame 1 of the noise superimposed voice data and passes it to the particle filter unit 32 (step S204). The particle filter means 32 uses a speech noise acoustic model corresponding to a beep sound and performs sampling from the acoustic model. In this case, since the target utterance is not included, sampling from the acoustic model corresponding to the target utterance is not performed. Then, the particle filter unit 32 updates the background noise particles and the beep sound particles using the extended Kalman filter. Also, the weight is calculated and normalized for each particle. Thereafter, the particle filter unit 32 applies Residual sampling or Metropolis-Hastings sampling.

Next, the noise component calculation means 33 converts the average ^ n _t ^{(k) of} each particle estimated by the particle filter means 32, the variance ^ Σ _nt ^(k) , the weight of each particle, and clean audio data. The noise component is calculated using the corresponding speech noise acoustic model. At this time, since the target utterance is not included in this frame 1, the noise component is calculated using the lower equation of equation (4) (step S206). Further, the calculated noise component is converted into an impulse response by using the MEL-warped IDCT as described above, and passed to the noise suppression unit 34. The noise suppression unit 34 generates the frame 1 of the clean sound data by convolving the impulse response with the frame 1 of the noise superimposed sound data, and passes it to the storage unit 22 (step S207). In this way, the particle filter noise suppression unit 21 sequentially performs noise suppression processing using the particle filter for each frame (steps S108, S203 to S209).

  The accumulating unit 22 sequentially accumulates each frame of clean audio data received from the particle filter noise suppressing unit 21 in a recording medium (not shown) (step S109). In this way, noise suppression processing is performed on the noise-superimposed speech data, and clean speech data can be obtained. This clean voice data may be used, for example, for the user to listen to, or may be used for voice recognition processing, as described in the second embodiment described later, or other It may be used for processing.

  As described above, according to the noise suppression apparatus 1 according to the present embodiment, the maximum likelihood label sequence of noise superimposed speech data is acquired using multiple model synthesis, and noise expanded to multiple model synthesis according to the label sequence. By performing suppression processing, even when a new type of noise is suddenly input, the type of the sudden noise can be specified by the recognized label, and the sudden input Noise can also be suppressed appropriately. This is not realized by the noise suppression method using only the particle filter. In addition, by using the particle filter, it is possible to dynamically perform noise estimation using information obtained from the input noise-superimposed speech data, so that it is possible to cope with noise fluctuations. This is not possible with the conventional noise suppression method disclosed in Non-Patent Documents 6 and 7. Thus, the noise suppression apparatus 1 according to the present embodiment can appropriately cope with sudden new noise input and noise fluctuation. Specific experimental results will be described in Embodiment 2.

(Embodiment 2)
A speech recognition apparatus according to Embodiment 2 of the present invention will be described with reference to the drawings. The speech recognition apparatus according to the present embodiment includes the noise suppression device according to Embodiment 1, and performs speech recognition processing on clean speech data after noise suppression by the noise suppression device.

  FIG. 10 is a block diagram showing the configuration of the speech recognition apparatus 2 according to this embodiment. The speech recognition device 2 according to the present embodiment includes a speech recognition acoustic model storage unit 41, a language model storage unit 42, and speech recognition dictionary information in addition to the components of the noise suppression device 1 according to the first embodiment. A storage unit 43, a voice recognition unit 44, and an output unit 45 are provided. As described above, the speech recognition apparatus 2 according to the present embodiment includes the noise suppression apparatus 1 according to the first embodiment. Configurations and operations other than the speech recognition acoustic model storage unit 41, the language model storage unit 42, the speech recognition dictionary information storage unit 43, the speech recognition unit 44, and the output unit 45 are the same as those in the first embodiment. The description is omitted.

  The acoustic model storage unit 41 for speech recognition stores an acoustic model related to speech data that is a target of speech recognition. This acoustic model is for speech recognition. An acoustic model for speech recognition is already known and will not be described in detail. The process in which an acoustic model is memorize | stored in the acoustic model memory | storage part 41 for speech recognition is not ask | required. For example, an acoustic model may be stored in the speech recognition acoustic model storage unit 41 via a recording medium, or an acoustic model transmitted via a communication line or the like may be stored in the speech recognition acoustic model storage unit. 41 may be stored.

  The storage in the acoustic model storage unit 41 for speech recognition may be temporary storage in the RAM of the acoustic model read from an external storage device or the like, or may be long-term storage. The acoustic model storage unit 41 for speech recognition can be realized by a predetermined recording medium (for example, a semiconductor memory, a magnetic disk, an optical disk, etc.).

  The language model storage unit 42 stores a language model related to a recognition target language for speech recognition. This language model is for speech recognition, and is, for example, a bigram language model or a trigram language model. Language models for speech recognition are already known and will not be described in detail. The process in which a language model is memorize | stored in the language model memory | storage part 42 is not ask | required. For example, the language model may be stored in the language model storage unit 42 via a recording medium, or the language model transmitted via a communication line or the like may be stored in the language model storage unit 42. It may be.

  Storage in the language model storage unit 42 may be temporary storage in a RAM or the like of a language model read from an external storage device or the like, or may be long-term storage. The language model storage unit 42 can be realized by a predetermined recording medium (for example, a semiconductor memory, a magnetic disk, an optical disk, etc.).

  The speech recognition dictionary information storage unit 43 stores speech recognition dictionary information used for speech recognition. The speech recognition dictionary information is information that associates phoneme strings with words, for example. The dictionary information for speech recognition is already known and will not be described in detail. The process in which information is stored in the dictionary information storage unit 43 for speech recognition does not matter. For example, the speech recognition dictionary information may be stored in the speech recognition dictionary information storage unit 43 via a recording medium, or the speech recognition dictionary information transmitted via a communication line or the like may be stored as speech. It may be stored in the recognition dictionary information storage unit 43.

  Storage in the speech recognition dictionary information storage unit 43 may be temporary storage in the RAM or the like of speech recognition dictionary information read from an external storage device or the like, or may be long-term storage. The voice recognition dictionary information storage unit 43 can be realized by a predetermined recording medium (for example, a semiconductor memory, a magnetic disk, an optical disk, or the like).

  The training data storage unit 11, the label storage unit 12, the speech noise acoustic model storage unit 14, the synthetic acoustic model storage unit 15, the dictionary information storage unit 17, the label language model storage unit 18, and the storage unit 22 store clean speech data. Of the recording media (not shown), the acoustic model storage unit 41 for speech recognition, the language model storage unit 42, and the dictionary information storage unit 43 for speech recognition, any two or more recording media may be realized by the same recording medium. It may well be realized by separate recording media.

The speech recognition unit 44 recognizes speech of the clean speech data generated by the noise suppression device 1 using an acoustic model, speech recognition dictionary information, and a language model. This voice recognition process is already known and will not be described in detail. For the speech recognition process, refer to the following document, for example.
Literature: Kiyohiro Shikano, Tatsuya Kawahara, Mikio Yamamoto, Katsunobu Ito, Kazuya Takeda, “IT Text Speech Recognition System”, Ohmsha, 2001

  The output unit 45 outputs the voice recognition result by the voice recognition unit 44. This voice recognition result is, for example, text data. Here, the output may be, for example, display on a display device (for example, a CRT or a liquid crystal display), transmission via a communication line to a predetermined device, printing by a printer, or output to a recording medium. It may be accumulated or delivered to another component. The output unit 45 may or may not include an output device (for example, a display device or a printer). The output unit 45 may be realized by hardware, or may be realized by software such as a driver that drives these devices.

  Next, the operation of the noise suppression apparatus according to the present embodiment will be described using the flowchart of FIG. In the flowchart of FIG. 11, processes other than steps S301 to S303 are the same as those of the flowchart of FIG. 4 of the first embodiment, and the description thereof is omitted.

  (Step S301) The speech recognition unit 44 determines whether it is time to perform speech recognition processing. If it is time to perform the speech recognition process, the process proceeds to step S302; otherwise, the process returns to step S101.

  (Step S302) The speech recognition unit 44 is stored in the acoustic model stored in the speech recognition acoustic model storage unit 41, the language model stored in the language model storage unit 42, and the speech recognition dictionary information storage unit 43. The voice recognition process is performed on the clean voice data stored by the storage unit 22 using the voice recognition dictionary information.

(Step S303) The output unit 45 outputs a speech recognition result obtained by performing the speech recognition processing by the speech recognition unit 44. Then, the process returns to step S101.
Note that the processing is ended by powering off or interruption for aborting the processing in the flowchart in FIG.

  Specific examples of the speech recognition processing are already known, and specific examples other than the speech recognition processing are the same as those in the first embodiment.

  Next, an experimental example of the speech recognition apparatus 2 according to this embodiment will be described. In this experimental example, data recorded while a nurse performed actual work in a hospital was used as training data and noise-superimposed speech data. Specifically, the data for the first day was used as noise superimposed voice data, and the data for the second day was used as training data used for model learning. The training label information corresponding to the training data is created manually based on the training data.

  FIG. 12 is a table showing detailed experimental conditions. Data included in the training data and the noise-superimposed speech data has a length of 10 seconds and includes a target speech. The data was recorded at a sampling frequency of 32 kHz and 16 bits at a hospital and then down-sampled to 16 kHz. Because of the work shift, eight women were evaluated as training data. A tool such as a speech recognizer includes ATRASR large vocabulary speech recognition system Ver. 3.6 was used. For extraction of features used for noise suppression and learning of GMM, use HTK Ver. 3.3 was used. As the feature value, the output (FBANK) of the 24th order log mel filter bank was used. For noise label recognition, an MFCC model obtained by conversion from the FBANK model was used. A speaker adaptive GMM was used as a clean speech model. For other speech and noise models, GMMs with a 4, 8, 12 mixed distribution were used. As the covariance matrix, a diagonal covariance matrix was used. Thirty-two types of noise models were obtained from the learning data, and the number of synthesized models including synthesized models was 194. In the synthesis model, the maximum number of models synthesized into one model was three. The background noise was estimated for each noise-superimposed speech data and synthesized into all models. These 194 models were used as multi-label dictionary entries. The unknown multi-label rate was 3.77%.

  Regarding the particle filter, 300 particles were used in the conventional method (see Non-Patent Document 5 described above), and 110 particles were used in the method according to the present embodiment. This is a setting in which the noise suppression processing is approximately the same (Intel Pentium (registered trademark)-Real Time Factor is about 2.5 in the measurement at D3.2 GHz). In the method according to the present embodiment, the processing time for noise label recognition is also included. A tool such as a speech recognizer includes ATRASR large vocabulary speech recognition system Ver. 3.6 was used. MDL-SSS (see the following document 1) was used for the structure learning of the acoustic model for speech recognition. In this experiment, only the female speaker was used, so only the 5 mixed distribution female voice model created by re-learning was used. As a speaker adaptation method, MAP-VFS (refer to the following document 2) was used for the speech recognition system acoustic model.

  Reference 1: T. Jitsuhiro, T .; Matsui, S .; Nakamura, “Automatic generation of non-uniform HMM topologies based on the MDL criteria”, IEICE Trans. on Information and Systems, vol. E87-D, no. 8, p. 2121-2129, 2004

  Reference 2: M.M. Tonomura, T .; Kosaka, S .; Matsunaga, “Speaker adaptation based on transfer vector field smoothing using maximum a posteriori probabilistic estimation, Computer Speech and Language. 10, p. 117-132, 1996

  FIG. 13 shows an average of the total number of distributions of the noise suppression model used in the noise suppression method (MM-NS method) for performing label recognition for the evaluation speakers. FIG. 13 shows that the number of distributions increases exponentially with respect to the number of mixtures in each noise model. Considering that the total number of distributions included in the acoustic model for speech recognition used in this experimental example is 10,460, it can be said that the numbers are very large.

  FIG. 14 shows word recognition rates of the conventional method and the method according to the present embodiment. “SM-NS” indicates a “Single-Model Noise Suppression” method using a single distribution as a noise model. As this single distribution, the one estimated from the start 100 ms of each input voice was used. “PF” indicates a conventional method using a particle filter (see Non-Patent Document 5). This method did not obtain better accuracy than baseline for this task. This is because many insertion errors have occurred, and it is considered difficult to track for sudden noise. “PF-MM-NS (4 mix.)”, “PF-MM-NS (8 mix.)”, And “PF-MM-NS (12 mix.)” Have the number of mixed distributions of each noise model in the proposed method. The cases of 4, 8, and 12 are shown. These patterns have the same accuracy of 1% for the conventional MMNS, “MM-NS (4 mix.)”, “MM-NS (8 mix.)”, “MM-NS (12 mix.)” Using the same model. It exceeded the degree. “PF-MM-NS (12mix.)” Obtained an error improvement rate of 12.3% with respect to baseline. Also, “PF-MM-NS (4 mix.)” And “MM-NS (8 mix.)”, Or “PF-MM-NS (8 mix.)” And “MM-NS (12 mix.)” Are compared. Thus, it was found that the method according to the present embodiment can obtain the same level of performance with a small number of distributions (28% and 46%, respectively).

  Next, in order to see the effect of the proposed method in more detail, two intermediate processing patterns were evaluated. First, in order to see the effect of only speech segment detection in the noise suppression method for performing label recognition, speech segment detection by label recognition is performed and noise suppression processing is performed outside the speech segment, but nothing is performed in the speech segment. That is, “MM-NS (4 mix., VAD only)” when noise suppression is not performed. The other is that the particle filter of the method “PF-MM-NS (4mix.)” According to the present embodiment does not use particles according to noise, that is, the sequential noise is generated only with particles obtained from the background noise model. When estimating / suppressing, it is “PF (BG) -MMNS (4mix.)”. The number of particles at this time was set to 110, which is the same as the proposed method “PF-MM-NS (4mix.)”.

  FIG. 15 shows the word recognition rate for each pattern. Other than the above two results are the same as those included in FIG. In “MM-NS (4mix., VAD only)”, there is an improvement from “Baseline (w / o NS)” by performing speech section detection, but “MMNS (4mix.)” That performs noise suppression processing. It was less accurate. It can be seen that there is an effect of noise suppression by multiple models. “PF (BG) -MM-NS (4mix.)” Obtained slightly higher accuracy than “MM-NS (4mix.)” By performing successive noise estimation, but the proposed method “PF-MM-NS ( 4 mix.) ”. In the particle filter, it can be said that there is an effect of using a model corresponding to the noise type as the prior distribution of particles.

  As described above, the multi-model noise suppression method (MM-NS) that has been proposed in the past can handle a plurality of types of noises recorded simultaneously with audio data in an actual environment. However, since noise suppression is performed based on a noise model obtained from learning data in advance and a composite model obtained from the combination, it was not possible to cope with combinations and unknown noise distribution that could not be obtained with learning data. . Thus, in the method according to the present embodiment, a multiple model noise suppression method based on a particle filter has been proposed. By using the particle filter, it is possible to estimate the noise distribution from the input noise superimposed voice data. However, in order to deal with sudden noise, the label recognition result is used, and when new noise is detected, new particles are sampled using the noise model as a prior distribution. As long as the same type of noise continues, it is estimated by the extended Kalman filter, and when it can no longer be detected, the particle is deleted. This makes it possible to perform noise estimation dynamically using a priori knowledge obtained from learning data and using information obtained from noise-superimposed speech data. From the experimental results, it was found that the method according to the present embodiment can obtain higher accuracy than the noise suppression method that performs the conventional label recognition using the same noise model. In addition, detailed comparison experiments have shown the effect of detecting speech intervals in the noise suppression method that performs label recognition, the effect of using multiple noise models, and the effect of using models corresponding to noise as the prior distribution of particles.

  As described above, according to the speech recognition apparatus 2 according to the present embodiment, since speech recognition is performed using the clean speech data in which noise is suppressed by the noise suppression apparatus 1 described in Embodiment 1, higher word recognition is possible. Rate can be obtained.

  In the present embodiment, the configuration in which the speech recognition apparatus 2 includes the storage unit 22 has been described. However, the speech recognition apparatus 2 does not include the storage unit 22 and the clean speech data that has been noise-suppressed is stored in the speech recognition unit 44. You may pass directly to.

  In each of the above embodiments, the noise suppression device 1 and the speech recognition device 2 may not include the training data storage unit 11, the label storage unit 12, the model generation unit 13, and the label language model generation unit 16. When the noise suppression device 1 or the like does not include those components, for example, based on training data and training label information outside the device, a speech noise acoustic model, a synthetic acoustic model, a synthetic acoustic model, a label language model, It is assumed that dictionary information is generated, and the generated model and the like are accumulated in the speech noise acoustic model storage unit 14, the synthesized acoustic model storage unit 15, the dictionary information storage unit 17, and the label language model storage unit 18. The process in which a model etc. is memorize | stored in each memory | storage part is not ask | required. However, the generation of the synthesized acoustic model may be performed in the noise suppression device 1 or the like. In that case, the noise suppression device 1 or the like includes a model generation unit 13, and the model generation unit 13 synthesizes the speech noise acoustic model stored in the speech noise acoustic model storage unit 14 to synthesize the speech noise acoustic model. An acoustic model is generated and stored in the synthesized acoustic model storage unit 15. Also, part or all of the label language model may be generated manually as described above.

  Further, although cases have been described with the above embodiments where label recognition is performed using a label language model, label recognition may be performed without using a label language model. This is a so-called “No Grammar” method. In that case, the noise suppression device 1 and the speech recognition device 2 may not include the label language model storage unit 18, and the label recognition unit 20 may recognize the label without using the label language model. Good.

  In each of the above-described embodiments, the case where the noise suppression device 1 and the speech recognition device 2 are stand-alone has been described. However, the noise suppression device 1 and the speech recognition device 2 may be stand-alone devices. It may be a server device in a client system. In the latter case, the output unit or the reception unit receives an input or outputs a screen via a communication line.

  In each of the above embodiments, each processing or each function may be realized by centralized processing by a single device or a single system, or distributed processing by a plurality of devices or a plurality of systems. May be realized.

  Also, in each of the above embodiments, information related to processing executed by each component, for example, each component received, acquired, selected, generated, transmitted, or received Information and information such as threshold values, mathematical formulas, addresses, etc. used by each component in processing are retained temporarily or over a long period of time on a recording medium (not shown) even if not explicitly stated in the above description. May be. Further, the storage of information in the recording medium (not shown) may be performed by each component or a storage unit (not shown). Further, reading of information from the recording medium (not shown) may be performed by each component or a reading unit (not shown).

  In each of the above embodiments, when two or more components included in the noise suppression device 1 or the speech recognition device 2 include a communication device or an input device, the two or more components are physically single. You may have devices or you may have separate devices.

  In each of the above embodiments, each component may be configured by dedicated hardware, or a component that can be realized by software may be realized by executing a program. For example, each component can be realized by a program execution unit such as a CPU reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory. In addition, the software which implement | achieves the noise suppression apparatus 1 in the said embodiment is the following programs. In other words, this program is a kind of training data that includes a computer that accepts noise-superimposed speech data, which is speech data on which noise is superimposed, and training speech data and training noise data. A speech noise acoustic model stored in a speech noise acoustic model storage unit in which a plurality of speech noise acoustic models that are acoustic models of one type of noise data or a plurality of speech noise acoustic models is stored, speech data and noise data included in the training data A synthetic acoustic model stored in a synthetic acoustic model storage unit that stores a synthetic acoustic model that is an acoustic model in which two or more of them are synthesized, and identifies the type of speech and noise superimposed in the training data Dictionary information that is information for associating a label that is information to be performed with the speech noise acoustic model or the synthetic acoustic model Using the dictionary information stored in the stored dictionary information storage unit, a label recognition unit that recognizes a label corresponding to the noise-superimposed speech data for each frame, and an acoustic corresponding to the label recognized by the label recognition unit By sampling the particles from the model and updating the sampled particles, this is to function as a particle filter noise suppression unit that generates clean audio data in which noise of the noise superimposed audio data is suppressed.

  In the program, the functions realized by the program do not include functions that can be realized only by hardware. For example, functions that can be realized only by hardware such as a modem and an interface card in a reception unit that receives information and an output unit that outputs information are not included in at least the functions realized by the program.

  Further, this program may be executed by being downloaded from a server or the like, and a program recorded on a predetermined recording medium (for example, an optical disk such as a CD-ROM, a magnetic disk, a semiconductor memory, or the like) is read out. May be executed by

  Further, the computer that executes this program may be singular or plural. That is, centralized processing may be performed, or distributed processing may be performed.

  FIG. 16 is a schematic diagram illustrating an example of an external appearance of a computer that executes the program and realizes the noise suppression device and the speech recognition device according to the embodiment. The above-described embodiment is realized by computer hardware and a computer program executed on the computer hardware.

  In FIG. 16, a computer system 100 includes a computer 101 including a CD-ROM (Compact Disk Read Only Memory) drive 105 and an FD (Flexible Disk) drive 106, a keyboard 102, a mouse 103, and a monitor 104.

  FIG. 17 is a diagram illustrating a computer system. 17, in addition to the CD-ROM drive 105 and the FD drive 106, a computer 101 includes a CPU (Central Processing Unit) 111, a ROM (Read Only Memory) 112 for storing a program such as a bootup program, A CPU (Random Access Memory) 113 that is connected to the CPU 111 and temporarily stores application program instructions and provides a temporary storage space, a hard disk 114 that stores application programs, system programs, and data, a CPU 111 and a ROM 112. Etc. to each other. The computer 101 may include a network card (not shown) that provides connection to the LAN.

  A program that causes the computer system 100 to execute the functions of the noise suppression device and the speech recognition device according to the above-described embodiment is stored in the CD-ROM 121 or the FD 122 and inserted into the CD-ROM drive 105 or the FD drive 106. It may be transferred to the hard disk 114. Instead, the program may be transmitted to the computer 101 via a network (not shown) and stored in the hard disk 114. The program is loaded into the RAM 113 at the time of execution. The program may be loaded directly from the CD-ROM 121, the FD 122, or the network.

  The program does not necessarily include an operating system (OS) or a third-party program that causes the computer 101 to execute the functions of the noise suppression device and the speech recognition device according to the above-described embodiment. The program may include only a part of an instruction that calls an appropriate function (module) in a controlled manner and obtains a desired result. How the computer system 100 operates is well known and will not be described in detail.

  Further, the present invention is not limited to the above-described embodiment, and various modifications are possible, and it goes without saying that these are also included in the scope of the present invention.

  As described above, according to the noise suppression device and the like according to the present invention, noise can be removed from speech with superimposed noise, which is useful as a noise suppression device and a speech recognition device.

1 is a block diagram showing the configuration of a noise suppression device according to Embodiment 1 of the present invention. The block diagram which shows the structure of the particle filter noise suppression part of the noise suppression apparatus by the embodiment The figure for demonstrating the outline | summary of the particle filtering of the noise suppression apparatus by the embodiment Flowchart showing the operation of the noise suppression apparatus according to the embodiment Flowchart showing the operation of the noise suppression apparatus according to the embodiment The figure which shows an example of the training label information in the embodiment The figure which shows an example of the dictionary information in the embodiment The figure for demonstrating the label recognition in the embodiment The figure which shows an example of the result of the label recognition in the same embodiment Block diagram showing the configuration of a speech recognition apparatus according to Embodiment 2 of the present invention. The flowchart which shows operation | movement of the speech recognition apparatus by the embodiment The figure which shows an example of the experimental condition in the embodiment The figure for demonstrating the average total distribution number in the noise model in the embodiment The figure which shows an example of the result of the word recognition rate in the embodiment The figure for demonstrating the comparison of the effect of the PF-MM-NS method in the embodiment Schematic diagram showing an example of the appearance of the computer system in the embodiment The figure which shows an example of a structure of the computer system in the embodiment

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Noise suppression apparatus 2 Speech recognition apparatus 11 Training data memory | storage part 12 Label memory | storage part 13 Model production | generation part 14 Speech noise acoustic model memory | storage part 15 Synthetic acoustic model memory | storage part 16 Label language model production | generation part 17 Dictionary information memory | storage part 18 Label language model memory | storage Unit 19 accepting unit 20 label recognizing unit 21 particle filter noise suppressing unit 22 accumulating unit 31 feature quantity extracting unit 32 particle filter unit 33 noise component calculating unit 34 noise suppressing unit 41 acoustic model storage unit for speech recognition 42 language model storage unit 43 audio Dictionary information storage unit for recognition 44 Voice recognition unit 45 Output unit

Claims (5)

Speech noise acoustic model storage unit for storing one type of speech data included in training data including training speech data and training noise data or a plurality of speech noise acoustic models that are acoustic models of one type of noise data When,
A synthetic acoustic model storage unit that stores a synthetic acoustic model that is an acoustic model in which two or more of voice data and noise data included in the training data are synthesized;
A dictionary information storage unit for storing dictionary information that is information for associating the speech noise acoustic model or the synthetic acoustic model with a label that is information for identifying the type of speech and noise superimposed in the training data;
A reception unit that receives noise-superimposed voice data that is voice data on which noise is superimposed;
A label recognition unit that recognizes a label corresponding to the noise-superimposed speech data for each frame using the speech noise acoustic model, the synthetic acoustic model, and the dictionary information;
Particle filter noise suppression for generating clean voice data in which noise of the noise superimposed voice data is suppressed by sampling particles from an acoustic model corresponding to the label recognized by the label recognition unit and updating the sampled particles And a noise suppression device. The particle filter noise suppression unit is
Feature amount extraction means for extracting a feature amount for each frame of the noise-superimposed speech data;
From the acoustic model corresponding to the label recognized by the label recognition unit, using the feature amount corresponding to the noise superimposed speech data and the speech noise acoustic model or the synthetic acoustic model corresponding to the label recognized by the label recognition unit. Particle filter means for sampling a plurality of particles, updating the plurality of sampled particles, and calculating the weight of each updated particle;
Noise component calculating means for calculating a noise component for each frame using the particles updated by the particle filter means and the weights calculated by the particle filter means;
The noise suppression apparatus according to claim 1, further comprising: a noise suppression unit that removes the noise component calculated by the noise component calculation unit from the noise-superimposed speech data and acquires clean speech data. The noise suppression device according to claim 1 or 2,
An acoustic model storage unit for speech recognition in which an acoustic model related to speech data to be speech-recognized is stored;
A voice recognition dictionary information storage unit for storing voice recognition dictionary information used in voice recognition;
A language model storage unit that stores a language model related to a recognition target language for speech recognition;
Clean speech data generated by the noise suppression device, a speech recognition unit that recognizes speech using the acoustic model, the dictionary information for speech recognition, and the language model;
A speech recognition apparatus comprising: an output unit configured to output a speech recognition result obtained by the speech recognition unit. An accepting step for receiving noise-superimposed voice data, which is voice data on which noise is superimposed;
Speech noise acoustic model storage unit for storing one type of speech data included in training data including training speech data and training noise data or a plurality of speech noise acoustic models that are acoustic models of one type of noise data Is stored in a synthesized acoustic model storage unit that stores a synthesized acoustic model that is an acoustic model in which two or more types of speech data and noise data included in the training data are synthesized. A dictionary in which dictionary information is stored as information that associates the speech noise acoustic model or the synthetic acoustic model with a label that is information identifying the type of speech and noise superimposed in the training data Using the dictionary information stored in the information storage unit, the label corresponding to the noise superimposed speech data is recognized for each frame. And the label recognition step that,
Particle filter noise suppression for generating clean voice data in which noise of the noise superimposed voice data is suppressed by sampling particles from an acoustic model corresponding to the label recognized by the label recognition unit and updating the sampled particles And a noise suppression method. Computer
A reception unit that receives noise-superimposed voice data that is voice data on which noise is superimposed;
Speech noise acoustic model storage unit for storing one type of speech data included in training data including training speech data and training noise data or a plurality of speech noise acoustic models that are acoustic models of one type of noise data Is stored in a synthesized acoustic model storage unit that stores a synthesized acoustic model that is an acoustic model in which two or more types of speech data and noise data included in the training data are synthesized. A dictionary in which dictionary information is stored as information that associates the speech noise acoustic model or the synthetic acoustic model with a label that is information identifying the type of speech and noise superimposed in the training data Using the dictionary information stored in the information storage unit, the label corresponding to the noise superimposed speech data is recognized for each frame. And the label recognition unit that,
Particle filter noise suppression for generating clean voice data in which noise of the noise superimposed voice data is suppressed by sampling particles from an acoustic model corresponding to the label recognized by the label recognition unit and updating the sampled particles Program to function as a part.