JP6189818B2 - Acoustic feature amount conversion device, acoustic model adaptation device, acoustic feature amount conversion method, acoustic model adaptation method, and program - Google Patents

Description

  The present invention relates to a technique for converting an acoustic feature amount used for acoustic model learning when speech recognition using an acoustic model is adapted to various recognition target tasks.

  Patent Document 1 describes a technique for adapting an acoustic model to a task that is a target of speech recognition (hereinafter referred to as a recognition target task) in order to ensure a practical level of performance in speech recognition. Here, the recognition target task is a task having different acoustic characteristics such as a speaker, a noise type, and how to speak with respect to the original acoustic model.

  In general, the performance of speech recognition varies depending on the amount of learning data of the task to be recognized. That is, in a situation where the learning data of the task to be recognized does not exist satisfactorily, a satisfactory recognition rate is often not obtained even if the acoustic model is adapted by the conventional technique. Therefore, usually, a sufficient amount of learning data is collected by sufficiently collecting the voices of the tasks to be recognized and transcribe the voices, but this requires enormous monetary and time costs. If the speech of the task to be recognized is sufficiently available, it is possible to collect learning data by transcription, but in the first place, a sufficient amount of speech is not available for every task. For example, there are some tasks where it is difficult to obtain a sufficient amount of speech, such as dialects and Japanese speaking English.

  Non-Patent Document 1 describes a method of artificially creating a speaker variation in learning data by executing a Warping Factor of Vocal Tract Length Normalization (VTLN) with a plurality of values. Note that VTLN is described in Non-Patent Document 2.

JP 2007-249051 A

N. Jaitly, G. E. Hinton, “Vocal Tract Length Perturbation (VTLP) improves speech recognition”, ICML Workshop on Deep Learning for Audio, Speech, and Language Processing, 2013. E. Eide, H. Gish, “A parametric approach to vocal tract length normalization”, ICASSP, 1996.

  However, the technique of Non-Patent Document 1 has two major problems. First, since VTLN is a linear conversion process, only very rough conversion can be performed. The second point is that VTLN itself aims to convert the voice quality of the speaker, and other conversions cannot be performed.

  An object of the present invention is to improve the recognition rate even when learning data cannot be obtained sufficiently by artificially creating learning data for adapting an acoustic model to a task to be recognized.

  In order to solve the above-mentioned problem, the acoustic feature quantity conversion device according to the first aspect of the present invention is such that the target acoustic feature quantity sequence extracted from the target voice signal related to the task to be recognized, the target voice signal, and the utterance content correspond to each other. A feature quantity extraction unit that generates a reference acoustic feature quantity sequence extracted from a reference speech signal to be collated, and collation in which the correspondence relationship between the target acoustic feature quantity series and the reference acoustic feature quantity series is collated based on the similarity for each feature quantity Using the feature amount matching unit that generates the verified target acoustic feature amount sequence and the verified reference acoustic feature amount sequence, and the verified target acoustic feature amount sequence and the verified reference acoustic feature amount sequence. Model generation unit that learns a conversion model that outputs an acoustic feature quantity sequence obtained by converting a characteristic feature based on a correspondence relationship between a target acoustic feature quantity sequence and a reference acoustic feature quantity sequence, and a reference sound using the conversion model Includes a pseudo feature amount generating unit for generating a pseudo acoustic features sequence obtained by converting the acoustic features of the characterizing quantity sequence, the.

  The acoustic model adaptation device according to the second aspect of the present invention includes an acoustic feature amount storage unit that stores a target acoustic feature amount sequence extracted from a target speech signal related to a task to be recognized, and a conversion generated by the acoustic feature amount conversion device. A pseudo-acoustic feature quantity storage unit for storing a pseudo-acoustic feature quantity sequence obtained by converting an acoustic feature of an acoustic feature quantity series extracted from an audio signal having a different acoustic characteristic from a task to be recognized using a model; An acoustic model learning unit that learns an acoustic model using the feature amount sequence and the pseudo acoustic feature amount sequence.

  The acoustic feature amount conversion technique of the present invention creates learning data in a pseudo manner by using feature amount conversion by a neural network even when learning data of a recognition target task is not sufficiently obtained. Is also used to adapt the acoustic model. As a result, an acoustic model adapted to the task to be recognized can be generated more than the acoustic model adapted using only a small amount of learning data originally available, and the recognition rate is improved.

FIG. 1 is a diagram illustrating a functional configuration of an acoustic feature quantity conversion device and an acoustic model adaptation device according to the first embodiment. FIG. 2 is a diagram illustrating a processing flow of the acoustic feature quantity conversion method and the acoustic model adaptation method according to the first embodiment. FIG. 3 is a diagram illustrating a functional configuration of the acoustic feature quantity conversion device and the acoustic model adaptation device of the second embodiment. FIG. 4 is a diagram illustrating a processing flow of the acoustic feature quantity conversion method and the acoustic model adaptation method according to the second embodiment.

  In the present invention, in order to solve the above-described problems of the prior art, a neural network is used for the acoustic feature value conversion processing. The neural network is described, for example, in “Ryohei Nakano,“ Basic Mathematics of Neural Information Processing ”, Mathematical Engineering, 2005 (Reference 1)”. Unlike VTLN, a neural network realizes nonlinear processing, so it can express very complex expressions. In addition to the voice quality conversion of the speaker, the neural network can deal with other acoustic features such as noise type and how to speak, and is more versatile than VTLN.

<Points of invention>
In the present invention, a situation is assumed in which the learning data in the task to be recognized is not a sufficient amount to adapt the acoustic model. In the present invention, the acoustic model is adapted according to the following flow.
(1) On the premise that there is a small amount of learning data B that was originally available for the recognition target task and a sufficient amount of learning data A that is not a recognition target task, the acoustic data of the learning data B from the acoustic feature amount of the learning data A A converter for converting to a feature value is generated. Here, the converter uses a neural network.
(2) A sufficient amount of pseudo learning data C obtained by converting the learning data A using the converter is created.
(3) Using the original learning data B and the pseudo learning data C, a learning process for adapting the acoustic model to the recognition target task is performed.

Hereinafter, embodiments of the present invention will be described in detail. In addition, the same number is attached | subjected to the component which has the same function in drawing, and duplication description is abbreviate | omitted.
[First embodiment]
A first embodiment learns a neural network that converts acoustic feature amounts when complete parallel data exists, and uses the neural network to create pseudo learning data and a method for converting acoustic feature amounts The acoustic model adaptation device performs adaptation of the acoustic model using the learning data. Parallel data refers to a set of two acoustic feature quantity series having the same utterance content but different acoustic features. Examples of the acoustic features include a speaker, a noise type, and how to speak.

  As shown in FIG. 1, the acoustic feature amount conversion apparatus 1 according to the first embodiment includes an input terminal 10, an audio signal acquisition unit 11, a label assignment unit 12, a feature amount extraction unit 13, a feature amount verification unit 14, and a conversion model generation. The unit 15, the pseudo feature amount generation unit 16, the audio signal storage unit 21, the feature amount storage unit 22, the conversion model storage unit 23, and the pseudo feature amount storage unit 24 are included.

  As shown in FIG. 1, the acoustic model adaptation device 2 according to the first embodiment includes, for example, an acoustic model learning unit 17 in addition to the components of the acoustic feature quantity conversion device 1. In FIG. 1, the configuration in which all the components of the acoustic feature quantity conversion device 1 are included in the acoustic model adaptation device 2 is illustrated, but the feature quantity storage unit 22 that stores the output of the external acoustic feature quantity conversion device 1 and A configuration including only the pseudo feature amount storage unit 17 is also possible.

  The acoustic feature quantity conversion device 1 and the acoustic model adaptation device 2 are special programs in a known or dedicated computer having a central processing unit (CPU), a main storage device (RAM), and the like, for example. Is a special device constructed by reading. For example, the acoustic feature quantity conversion device 1 and the acoustic model adaptation device 2 execute each process under the control of the central processing unit. The data input to the acoustic feature quantity conversion device 1 and the acoustic model adaptation device 2 and the data obtained in each process are stored in, for example, the main storage device, and the data stored in the main storage device is read as necessary. To be used for other processing. Moreover, at least a part of each processing unit of the acoustic feature quantity conversion device 1 and the acoustic model adaptation device 2 may be configured by hardware such as an integrated circuit.

  Each storage unit included in the acoustic feature quantity conversion device 1 and the acoustic model adaptation device 2 is, for example, a main storage device such as a RAM (Random Access Memory), a semiconductor memory element such as a hard disk, an optical disk, or a flash memory. The auxiliary storage device may be configured, or may be configured by middleware such as a relational database or a key / value store. Each storage unit included in the acoustic feature quantity conversion device 1 and the acoustic model adaptation device 2 may be logically divided, and may be stored in one physical storage device.

  With reference to FIG. 2, the processing procedure of the acoustic feature amount conversion method of the first embodiment will be described.

  In step S <b> 10, an acoustic signal as learning data is input to the input terminal 10 of the acoustic feature quantity conversion device 1. The learning data includes a speech signal related to the task to be recognized (hereinafter referred to as a target speech signal) and a speech signal (hereinafter referred to as a reference speech signal) having the same utterance content as the target speech signal but having different acoustic characteristics. . The audio signal to be input may be a sound collected from a human voice in real time by connecting a sound collecting means such as a microphone to the input terminal 10, or a voice recording function of an IC recorder or a smartphone in advance. Recording may be performed on a recording medium such as a non-volatile memory or a hard disk drive by a recording means such as that described above and may be input by being reproduced by a reproducing means connected to the input terminal 10.

  In step S11, the audio signal acquisition unit 11 converts an analog input audio signal into a digital signal. When a digital audio signal is input from the input terminal 10, the audio signal acquisition unit 11 may not be provided. The digital input voice signal is stored in the voice signal storage unit 21.

  In step S12, the label assigning unit 12 reads the target audio signal and the reference audio signal stored in the audio signal storage unit 21, and assigns a label representing the acoustic feature of each audio signal. The target audio signal and the reference audio signal to which the label is assigned are sent to the feature amount extraction unit 13. An acoustic feature is an attribute of a voice signal required by a recognition target task, and examples thereof include a speaker, a noise type, and how to speak. Examples of labeling methods include (1) a method in which a user designates a scene to use when recording audio in advance, (2) automatic acquisition by login authentication or an application used, and (3) automatic classification by automatic classification by clustering. There is an acquisition.

  (1) The method specified by the user is that when a voice is recorded, if it is a speaker, “who speaks (for example, personal attribute information such as gender, age, place of residence, language, personal name)”, noise If it is a type, the user designates “where to speak (for example, the use environment such as in a car, in the city, a conference room)”, etc., thereby giving a label of an acoustic feature to the target audio signal.

  (2) The automatic acquisition method by an application or the like is as follows. If it is a speaker, for example, a user's label is given by providing user login authentication before recording a sound. If it is a noise type, a label is given according to the type of application used. For example, it is conceivable to give labels such as in-vehicle noise for car navigation voice recognition, and noise emitted from a television for games using voice recognition. Furthermore, after obtaining the noise type from the application in use, it may be further classified according to the noise level. As the noise level, for example, an absolute value of a sound pressure level of noise measured by a sound level meter, an S / N ratio based on each recorded sound pressure level of recorded sound and noise, or the like can be used. Furthermore, time information such as the date and time when the audio signal was recorded and spatial information such as a location may be added to classify them finely.

  (3) As a method of automatic acquisition by clustering, speakers, recording environment, how to speak, etc. are clustered by, for example, the well-known K-means method, etc., and “Speaker 1”, “Speaker 2”, “Speaker” are used as labels. 3 ”,“ Recording environment 1 ”,“ Recording environment 2 ”,“ Recording environment 3 ”, and so on, and“ How to beat 1 ”,“ How to beat 2 ”,“ How to beat 3 ”, and so on. As acoustic features used for clustering, for example, 1 to 12 dimensions of Mel-Frequency Cepstrum Coefficient (MFCC) based on short-time frame analysis of speech signals, and ΔMFCC and ΔΔMFCC which are dynamic features Dynamic parameters such as power, Δ power, ΔΔ power, and the like are used. At this time, if clustering is performed for a speaker, for example, an average of the above-described acoustic features in the utterance section is extracted for each utterance section, and clustering is performed using the average. In this case, speakers with similar acoustic features may be classified into the same cluster, but since the acoustic features can be grouped as speakers with the same tendency, the performance of the feature transformation neural network described later Will not be affected. In the case of clustering for the noise type, for example, acoustic features are extracted from the sections other than the speech section (that is, sections representing the recording environment) in the same manner as in the case of the speaker, and are averaged in the sections other than the speech section. Clustering for. For example, the clustering method for speaking can be classified into a reading tone and a free speech tone, and the input speech is automatically classified using the GMM Supervector. . Clustering for this type of speaking is described in “T. Asami, R. Masumura, H. Masataki, S. Sakauchi,“ Read and Spontaneous Speech Classification Based on Variance of GMM Supervectors ”, Interspeech 2014, pp. 2375-2379, 2014. (Reference Document 2) ”. When clustering as described above is performed, a specific cluster is linked to the speaker, recording environment, or how to speak, so the utterance section that is highly similar to the cluster, sections other than the utterance section, and how to speak And a label for extracting an acoustic feature amount by the feature amount extraction unit 13 described later can be assigned.

  A combination of the labeling methods (1) to (3) above may be used to give a plurality of types of labels. For example, a speaker's label can be automatically acquired and given by user login authentication, a recording environment label can be given by designation by the user, and a turn-over label can be given by clustering.

  In step S13, the feature quantity extraction unit 13 extracts each acoustic feature quantity from the labeled target voice signal and the reference voice signal, and extracts a series of labeled acoustic feature quantities. The labeled target acoustic feature quantity sequence and the reference acoustic feature quantity series are stored in the feature quantity storage unit 22. As acoustic features to be extracted, for example, 1 to 12 dimensions of Mel-Frequency Cepstrum Coefficient (MFCC) based on short-time frame analysis of audio signals and their dynamic features ΔMFCC, ΔΔMFCC, etc. Dynamic parameters, power, Δ power, ΔΔ power, and the like are used. Moreover, you may perform a cepstrum mean normalization (CMN: Cepstral Mean Normalization) process with respect to MFCC. The acoustic feature amount to be extracted is not limited to MFCC or power, but a parameter used for speech recognition may be used.

  The feature quantity storage unit 22 accumulates the labeled target acoustic feature quantity series and the labeled reference acoustic feature quantity series. As described above, the labeled target acoustic feature quantity sequence is a labeled acoustic feature quantity extracted from the target speech signal recorded under an environment corresponding to the speaker, noise type, and how to beat according to the recognition target task. It is a series. The labeled reference acoustic feature quantity sequence is not a recognition target task, but is a series of labeled acoustic feature quantities extracted from a reference speech signal with high speech clarity and capable of acquiring a large amount of data. As the labeled reference acoustic feature quantity sequence, for example, it is conceivable to use a series of acoustic feature quantities based on a voice signal recorded in order to generate an original acoustic model used in voice recognition. In addition, it is desirable that the two acoustic feature quantity series include many utterances of the same word or utterances uttered with similar sounds but not the same word (for example, “genki” and “weather”). Further, it is assumed that the utterance contents in each acoustic feature quantity series are all known.

  In step S <b> 14, the feature amount matching unit 14 determines the correspondence between the labeled target acoustic feature amount sequence stored in the feature amount storage unit 22 and the reference acoustic feature amount sequence for each feature amount in units of short frames. Match on time series based on magnitude of degree. The verified target acoustic feature quantity sequence and the reference acoustic feature quantity sequence are sent to the conversion model generation unit 15. The target acoustic feature quantity series with the label and the reference acoustic feature quantity series have the same utterance content but different labels such as speaker, noise type, and manner of speaking. In general, even if the utterance content is the same, the length of the utterance time may be different. In order to learn a neural network, it is necessary to associate the same utterance content in units of frames. Therefore, before learning a neural network for converting acoustic features, it is necessary to collate on the time axis. . As a verification method, a known dynamic time expansion / contraction method and the like can be given. The dynamic time expansion and contraction method is described in “Seiichi Uchida,“ DP Matching Overview: Basics and Various Extensions ”, IEICE Technical Report, PRMU 2006-166, pp. 31-36, 2006 (Reference 3)”. Has been. Based on the DP path generated as a result of collation by this processing, for each feature quantity included in the two labeled acoustic feature quantity series, a time-series correspondence between highly similar feature quantities is obtained. .

  In step S <b> 15, the conversion model generation unit 15 uses the collated target acoustic feature amount sequence and the reference acoustic feature amount sequence to convert the acoustic features of the input acoustic feature amount sequence into the target acoustic feature amount sequence and the reference acoustic feature. A conversion model that outputs an acoustic feature amount sequence converted based on a correspondence relationship with the feature amount sequence is learned. In this embodiment, this conversion model is a neural network. Hereinafter, this conversion model is referred to as a feature amount conversion neural network. The feature quantity conversion neural network is stored in the conversion model storage unit 23. As a learning method of the neural network, for example, a known error back-propagation method or stochastic gradient descent method may be used. The error back-propagation method and the stochastic steepest gradient method are described in Masahiro Araki, “Introduction to Machine Learning with Free Software”, Morikita Publishing, 2014 ”.

  A specific learning method of the feature amount conversion neural network will be described. For example, in the case of a feature amount conversion neural network for converting speaker A to speaker B, speaker A's speech and speaker B's speech are converted into verified acoustic feature sequences, respectively, and then speaker A's verification is performed. The feature amount conversion neural network is learned by using the already-acquired acoustic feature amount sequence as an input, and outputting the collated acoustic feature amount sequence of the speaker B who has a corresponding relationship on the time axis with the input acoustic feature amount sequence. The feature quantity conversion neural network is similarly learned for other acoustic features (for example, noise type and how to beat). For example, in the case of a noise type, after converting an utterance under noise type A and an utterance under noise type B to a matched acoustic feature quantity sequence, learning those matched acoustic feature quantity sequences, Generate a feature conversion neural network. In the above description, a feature quantity conversion neural network is generated for each acoustic feature. However, a feature quantity conversion neural network that converts them in a composite manner may be generated. That is, a feature quantity conversion neural network that converts an acoustic feature quantity sequence uttered by speaker A under noise type A into an acoustic feature quantity sequence spoken by speaker B under noise type B may be generated. . In this case, the collated acoustic feature quantity series to which the labels of speaker A and noise type A are assigned is input, and the labels of speaker B and noise type B having a correspondence relationship on the time axis are assigned. The feature-value conversion neural network may be learned by using the matched feature-value series with collation as an output.

  In step S <b> 16, the pseudo feature quantity generation unit 16 converts an acoustic feature quantity sequence extracted from a speech signal having an acoustic feature different from that of the recognition target task, using a feature quantity conversion neural network stored in the conversion model storage unit 23. Then, the pseudo acoustic feature amount is generated. At that time, a feature quantity conversion neural network of a type matching the pseudo acoustic feature quantity to be output is selected for the inputted acoustic feature quantity series. For example, if an acoustic feature is to be converted from speaker A to speaker B, a feature amount conversion neural network that converts an acoustic feature from speaker A to speaker B is selected. The generated pseudo acoustic feature quantity sequence is stored in the pseudo feature quantity storage unit 24.

  Next, the processing procedure of the acoustic model adaptation method of the first embodiment will be described with reference to FIG.

  The feature quantity storage unit 22 stores a target acoustic feature quantity sequence generated by the above-described acoustic feature quantity conversion method from a target voice signal related to a recognition target task.

  The pseudo feature quantity storage unit 24 stores a pseudo acoustic feature quantity sequence obtained by converting an acoustic feature quantity sequence extracted from an audio signal having an acoustic feature different from that of the recognition target task by the above-described acoustic feature quantity conversion method.

  In step S <b> 17, the acoustic model learning unit 17 learns an acoustic model using the target acoustic feature amount sequence stored in the feature amount storage unit 22 and the pseudo acoustic feature amount sequence stored in the pseudo feature amount storage unit 24. To do. For example, GMM-HMM is used as an acoustic model in speech recognition. For example, Koichi Shinoda, “Speaker Adaptation Technology for Speech Recognition Using Probability Models” ", IEICE Transactions, J87-D-II (2), pp. 371-386, 2004 (Reference 4)".

  As described above, the acoustic feature quantity conversion apparatus and method according to the first embodiment provide an acoustic feature of an audio signal having an audio feature that is different from the recognition target task even if the audio signal related to the recognition target task cannot be sufficiently prepared. It is possible to prepare sufficient learning data by generating a pseudo acoustic feature amount by converting. Therefore, the recognition rate of the acoustic model adapted to the recognition target task is improved.

[Second Embodiment]
The second embodiment learns a neural network that converts acoustic feature amounts when perfect parallel data does not exist, and uses the neural network to create pseudo learning data and an acoustic feature amount conversion device and method The acoustic model adaptation device performs adaptation of the acoustic model using the learning data.

  In the first embodiment, the feature conversion neural network is learned using parallel data with the same utterance content but different acoustic features such as speaker, noise type, and how to speak. However, there are cases where such parallel data exists. Data sparseness problems can occur because they are rare and difficult to collect in large quantities, even if they exist. Therefore, in the second embodiment, consider the case of using non-parallel data. The correspondence on the time axis using the dynamic time expansion / contraction method or the like is premised on the same utterance content, that is, parallel data. Therefore, for non-parallel data, a state number (a state number on a hidden Markov model described later) for each analysis frame of each utterance is estimated in advance, and the feature number conversion neural network is generated by associating the state numbers with each other. . Therefore, in the second embodiment, the process of the feature amount matching unit is different from the first embodiment.

  As shown in FIG. 3, the acoustic feature value conversion device 3 according to the second embodiment includes an input terminal 10, an audio signal acquisition unit 11, a label assignment unit 12, a feature value extraction unit 13, a conversion model generation unit 15, and a pseudo feature value. The generation unit 16, the acoustic model learning unit 17, the speech signal storage unit 21, the feature amount storage unit 22, the conversion model storage unit 23, and the pseudo feature amount storage unit 24 are included as in the first embodiment, and the utterance forced alignment unit 18. And the feature-value collation part 19 is included, for example.

  With reference to FIG. 4, the acoustic feature-value conversion method of 2nd embodiment is demonstrated. Below, it demonstrates centering on difference with the above-mentioned 1st embodiment.

  In step S <b> 18, the utterance forced alignment unit 18 executes the forced alignment from the target acoustic feature amount sequence and the reference acoustic feature amount sequence stored in the feature amount storage unit 22, thereby aligning the target acoustic feature amount sequence that has been aligned. A reference acoustic feature quantity sequence is generated. The generated target acoustic feature quantity sequence and reference acoustic feature quantity series generated are sent to the feature quantity matching unit 19. Forced alignment is based on the assumption that the utterance content of the acoustic feature series is known, and performs speech recognition on the correct text that matches the utterance content, and observes state transitions in the recognition processing process. This is a process of assigning a hidden Markov model (HMM) state number corresponding to each feature amount. In speech recognition, a hidden Markov model is often used for phoneme recognition, and the state number is considered up to triphone. The triphone is a triple of phonemes including the phoneme relationship before and after the phonemes to be classified. In the triphone, for example, three phonemes such as “a-k-a” are considered as one state number. Note that a monophone is considered as one phoneme, and a biphone is considered as one phone number. Forced alignment is executed using a Viterbi algorithm or the like using correct text. Hidden Markov models and Viterbi algorithms for speech recognition are described in “Kano et al.,“ IT Text Speech Recognition System ”, Ohmsha, 2001”.

  In step S19, the feature amount matching unit 19 checks the aligned target acoustic feature amount sequence and the reference acoustic feature amount sequence with the state numbers assigned to them. The target acoustic feature quantity sequence and the reference acoustic feature quantity sequence that have been verified are sent to the conversion model generation unit 15. For example, in the aligned acoustic feature quantity sequence of speaker A and the aligned acoustic feature quantity sequence of speaker B, an analysis frame having a different utterance content but the same state number is output as a matched acoustic feature quantity sequence. . For example, when “weather” spoken by speaker A and “genki” spoken by speaker B are compared at the phoneme level, the utterance contents are different, but “g / e / ng / k / i” and “t / In “e / ng / k / i”, “ng” or the like is the same phoneme, and the phoneme relationship before and after is the same, so the same state number is assigned.

  The feature amount collating unit 19 can obtain two collated acoustic feature amount sequences in which the alignment of the state transitions is matched, although not the entire utterance. The alignment of the state transition is not necessarily the same physical quantity as the correspondence on the time axis, but can be handled in the same way as the correspondence on the time axis as a correspondence between the two acoustic feature quantity sequences. .

  As described above, the acoustic feature amount conversion apparatus and method according to the second embodiment uses the collated acoustic feature amount sequence in which the alignment of the state transitions matches by the forced alignment even when the parallel data cannot be sufficiently prepared. Therefore, a pseudo acoustic feature quantity sequence similar to the acoustic feature quantity conversion apparatus and method of the first embodiment can be obtained. Accordingly, as in the first embodiment, the recognition rate of the acoustic model adapted to the recognition target task is improved.

  The present invention is not limited to the above-described embodiment, and it goes without saying that modifications can be made as appropriate without departing from the spirit of the present invention. The various processes described in the above embodiment may be executed not only in time series according to the order of description, but also in parallel or individually as required by the processing capability of the apparatus that executes the processes or as necessary.

[Program, recording medium]
When various processing functions in each device described in the above embodiment are realized by a computer, the processing contents of the functions that each device should have are described by a program. Then, by executing this program on a computer, various processing functions in each of the above devices are realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. When executing the process, the computer reads a program stored in its own recording medium and executes a process according to the read program. As another execution form of the program, the computer may directly read the program from a portable recording medium and execute processing according to the program, and the program is transferred from the server computer to the computer. Each time, the processing according to the received program may be executed sequentially. Also, the program is not transferred from the server computer to the computer, and the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by the execution instruction and result acquisition. It is good. Note that the program in this embodiment includes information that is used for processing by an electronic computer and that conforms to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer).

  In this embodiment, the present apparatus is configured by executing a predetermined program on a computer. However, at least a part of these processing contents may be realized by hardware.

DESCRIPTION OF SYMBOLS 1, 3 Acoustic feature-value conversion apparatus 2 Acoustic model adaptation apparatus 10 Input terminal 11 Audio | voice signal acquisition part 12 Label provision part 13 Feature-value extraction part 14, 19 Feature-value collation part 15 Conversion model production | generation part 16 Pseudo-feature-value production | generation part 17 Acoustic Model learning unit 18 Utterance compulsory alignment unit 21 Speech signal storage unit 22 Feature amount storage unit 23 Conversion model storage unit 24 Pseudo feature amount storage unit

Claims (6)

The target acoustic feature quantity sequence extracted from the target speech signal including the unclean speech signal related to the task to be recognized, and the reference acoustic feature quantity extracted from the reference speech signal including the unclean speech signal corresponding to the target speech signal and the utterance content. A feature amount extraction unit that generates a sequence;
An utterance forced alignment unit that performs speech recognition based on a probability model for the target acoustic feature amount sequence and the reference acoustic feature amount sequence, and assigns a state number of the probability model;
A collated target acoustic feature sequence and a collated reference acoustic feature sequence, in which the correspondence relationship between the target acoustic feature sequence and the reference acoustic feature sequence is collated based on a phoneme sequence having the same transition of the state number. A feature amount matching unit to be generated;
Learning a conversion model that outputs an acoustic feature quantity sequence obtained by converting an acoustic feature of an input acoustic feature quantity based on the correspondence using the matched target acoustic feature quantity sequence and the matched reference acoustic feature quantity sequence A conversion model generation unit to
A pseudo feature generating unit that generates a pseudo acoustic feature amount sequence obtained by converting the acoustic feature of the acoustic feature amount sequence extracted from an audio signal having an acoustic feature different from the task using the conversion model;
An acoustic feature quantity conversion device including: The acoustic feature amount conversion apparatus according to claim 1 ,
The conversion model is a neural network that converts an acoustic feature of the input acoustic feature quantity based on a correspondence relationship between the target acoustic feature quantity sequence and the reference acoustic feature quantity series. An acoustic feature quantity storage unit for storing a target acoustic feature quantity sequence extracted from a target speech signal related to a task to be recognized;
A pseudo-acoustic feature amount obtained by converting an acoustic feature of an acoustic feature amount sequence extracted from an audio signal having a different acoustic feature from the task, using a conversion model generated by the acoustic feature-value conversion device according to claim 1 or 2. A pseudo-acoustic feature storage unit for storing a sequence;
An acoustic model learning unit that learns an acoustic model using the target acoustic feature amount sequence and the pseudo acoustic feature amount sequence;
An acoustic model adaptation device including: From the target sound feature amount sequence extracted from the target sound signal including the unclean sound signal related to the task to be recognized by the feature amount extraction unit and the reference sound signal including the unclean sound signal corresponding to the target sound signal and the utterance content A feature extraction step for generating an extracted reference acoustic feature sequence;
An utterance forced alignment unit performs speech recognition based on a probability model for the target acoustic feature quantity sequence and the reference acoustic feature quantity series, and assigns a state number of the probability model, and an utterance forced alignment step,
The feature amount matching unit matches the reference relationship with the matched target acoustic feature amount sequence in which the correspondence relationship between the target acoustic feature amount sequence and the reference acoustic feature amount sequence is matched based on the phoneme sequence in which the transition of the state number matches. A feature amount matching step for generating an acoustic feature amount sequence;
The conversion model generation unit converts an acoustic feature quantity sequence obtained by converting the acoustic feature of the input acoustic feature quantity based on the correspondence relationship using the matched target acoustic feature quantity series and the matched reference acoustic feature quantity series. A conversion model generation step for learning a conversion model to be output;
A pseudo feature quantity generating step in which the pseudo feature quantity generating unit generates a pseudo acoustic feature quantity sequence obtained by converting the acoustic features of the acoustic feature quantity series extracted from the audio signal having different acoustic characteristics from the task using the conversion model. When,
An acoustic feature amount conversion method including: The acoustic feature quantity storage unit stores a target acoustic feature quantity sequence extracted from the target speech signal related to the task to be recognized,
The acoustic feature of the acoustic feature quantity sequence extracted from an audio signal having an acoustic feature different from that of the task using the transformation model generated by the acoustic feature quantity transformation method according to claim 4 in the pseudo acoustic feature quantity storage unit. The converted pseudo acoustic feature quantity sequence is stored,
An acoustic model learning unit that learns an acoustic model using the target acoustic feature quantity sequence and the pseudo acoustic feature quantity series; and
An acoustic model adaptation method including: A program for causing a computer to function as the acoustic feature quantity conversion device according to claim 1 or 2 or the acoustic model adaptation device according to claim 3 .

Images