JP6267667B2 - Learning data generating apparatus, method and program - Google Patents

Description

  The present invention relates to a learning data generation technique for a speaker feature extraction model that can perform speaker recognition with high accuracy.

  Non-Patent Document 1 discloses a method for calculating a speaker feature vector used for speaker recognition from an input voice signal. Divides the input speech signal (usually the speech signal of the section called “speech” that utters one sentence) into several tens of msec acoustic analysis frames, and extracts the acoustic feature vector of each acoustic analysis frame Then, an acoustic feature vector sequence arranged in time order is created, and a speaker feature vector w is calculated from the acoustic feature vector sequence by the following equation (1). The following equation (1) corresponds to equation (13) of Non-Patent Document 1.

w = (I + T'Σ ^-1 NuT) ^-1 TΣ ^-1 Fu (1)
I represents a unit matrix, and “′” represents transposition of the matrix. Nu and Fu are a zero-order statistic and a first-order statistic calculated with respect to a predetermined mixed normal distribution using the input acoustic feature vector sequence, respectively. T and Σ are parameters of the speaker feature extraction model and are learned before the speaker feature extraction.

  Since speaker feature vectors obtained from the same speaker's utterance have similar properties (value of cosine similarity increases), speaker recognition can be performed using the speaker feature vectors.

Tetsuji Ogawa and Sayaka Shioda, “Speaker recognition using i-vector,” The Acoustical Society of Japan, Vol. 70, No. 6, pp.332-339, June 2014.

  The conventional technique described in Non-Patent Document 1 requires a learning utterance set (hereinafter referred to as “learning set”) for estimating parameters T and Σ of a speaker feature extraction model. In order to learn a speaker feature extraction model with high accuracy, it is desirable that the learning set includes various speakers, various recording devices, and utterances recorded in various ambient noise environments. As the number of utterances increases, the time required for the learning process increases and the amount of memory used also increases. Therefore, there is an upper limit on the size of a learning set that can be practically used (usually about tens of thousands of utterances). Therefore, tens of thousands of utterances are randomly selected from a large audio data set (hereinafter referred to as “maternal set”) consisting of millions of utterances usually recorded in various speakers, recording devices, and ambient noise environments. Use as a learning set. It is possible to select using a label if each utterance of the mother set is given a label indicating what the speaker, recording device, and ambient noise are, but it is a task to give a label to a large audio data set Since the cost is high and it is not realistic to actually perform, a learning set is created by random selection.

  However, the learning set obtained by random selection does not always fully cover various speakers, recording devices, and ambient noise included in the mother set. When the diversity of speakers included in the learning set is low, the parameters T and Σ of the speaker feature extraction model to be learned do not become good values, and the speaker feature vectors of different speakers are similar. There is a possibility that the speaker recognition performance is degraded. Even when the diversity of recording devices included in the learning set is low, speaker feature vectors may be different if the recording device is different even for the same speaker, and speaker recognition performance may deteriorate. Similarly, when the diversity of ambient noises included in the learning set is low, speaker feature vectors may be separated if the ambient noise is different even for the same speaker, which may reduce speaker recognition performance. .

  An object of the present invention is to provide a learning data generating apparatus, method, and program for generating learning data including various speakers, recording devices, and ambient noise.

  A learning data generation device according to an aspect of the present invention includes an acoustic feature quantity extraction unit that extracts an acoustic feature quantity vector group from an audio signal of each utterance included in a matrix set, and a predetermined number of mixtures with respect to the acoustic feature quantity vector group Using the acoustic feature vector group as a matrix, a mixed normal distribution fitting unit that obtains a mixed normal distribution by fitting the mixed normal distribution of M and the M normal distributions constituting the obtained mixed normal distribution as components. The component content calculation unit that calculates the content of each component in the set and the composition ratio of each component in the learning set from the utterances included in the parent set based on the content of each component in the parent set Learning data by selecting the utterance so that it is close to the composition ratio of each component in And a, a speech selection unit for generating a.

  Learning data including various speakers, recording devices, and ambient noise can be generated.

The block diagram for demonstrating the example of a learning data generation apparatus. The flowchart for demonstrating the example of the learning data generation method.

[Technical background]
One of the ideas of the present invention is to calculate a diversity score of an utterance set that represents the diversity of acoustic properties (speakers, recording devices, and ambient noise) included in the utterance set, and to increase the diversity score. Is selected from the mother set and used as a learning set.

  The diversity score is calculated based on the idea that “if the utterance set includes all the components (parts) that make up the speech of the mother set, the diversity score is high”. First, the matrix set is decomposed into components (each normal distribution of the mixed normal distribution) by applying a mixed normal distribution to the acoustic feature vectors of all frames of the matrix set. Next, how much component each utterance in the matrix set contains is calculated based on the likelihood of each normal distribution of the mixed normal distribution. When one utterance is selected as a learning set from the mother set, components included in the selected utterance are added to the learning set. If the composition ratio of the components included in the learning set created by selecting several utterances is close to the component ratio of the entire mother set, it can be said that the learning set includes all the components of the mother set. Therefore, the diversity score D (U) of the learning set U is calculated by the following equation that increases when the component ratio of each component included in the learning set is close to the component ratio of the matrix set component.

M is the number of mixtures in the mixed normal distribution (number of components), w _i is the ratio of the i-th component in the matrix set, and _fi u is the content of the i-th component in the learning set U.

  As described in the section “Problems to be Solved by the Invention”, the learning set of the speaker feature extraction model has an upper limit of size (tens of thousands of utterances) due to restrictions on calculation time and memory usage. Assuming that the upper limit of the number of utterances in the learning set is C, various speakers and recordings are selected by selecting the learning set U from the mother set so that D (U) is as large as possible within a range satisfying | U | ≦ C. It is possible to select a learning set including equipment and ambient noise from the parent set. | U | is the number of utterances included in the learning set U.

  However, the number of combinations for selecting tens of thousands of utterances from a matrix set consisting of millions of utterances is enormous, and it is realistic to calculate the value of D (U) for all utterance combinations and find the maximum U. It is impossible in time. Therefore, the learning set U is selected in a realistic processing time by adding one utterance from the mother set to the learning set U until | U | = C by the greedy method. Since the diversity score D (U) is a submodular function, it is possible to obtain U that approximately maximizes D (U) by selecting an utterance by the greedy method.

[Embodiment]
Examples of embodiments of the present invention will be described below with reference to the drawings.

  The learning data generation device includes, for example, an acoustic feature quantity extraction unit 101, a mixed normal distribution fitting unit 102, and a component content calculation unit 103. The learning data generation method is realized by the learning data generation apparatus performing each step process illustrated in FIG.

<Sound Feature Extraction Unit 101>
Input: A matrix set is input to the acoustic feature quantity extraction unit 101.

Output: acoustic feature vector group of each utterance in the matrix set (to mixed normal distribution fitting unit 102 and component content calculation unit 103)
Process: The acoustic feature quantity extraction unit 101 extracts an acoustic feature quantity vector group from the speech signal of each utterance included in the input matrix set (step S1). The extracted acoustic feature vector group of each utterance in the matrix set is output to the mixed normal distribution fitting unit 102 and the component content calculation unit 103.

  In the extraction of the acoustic feature vector group, the speech signal of each utterance is divided into acoustic analysis frames of several tens of msec, and the acoustic feature vector is extracted from each acoustic analysis frame, thereby obtaining the acoustic feature vector group. The acoustic feature vector of each frame is a real value vector and may be extracted by any existing method such as MFCC or LPC cepstrum.

<Mixed normal distribution fitting unit 102>
Input: acoustic feature vector group of each utterance in the mother set (from the acoustic feature extraction unit 101), number of mixtures M
Output: mixed normal distribution (component content calculation unit 103 and mixed weight of each normal distribution to utterance selection unit 104)
Processing: The mixed normal distribution fitting unit 102 applies the mixed normal distribution of the number M of the input mixtures to the acoustic feature vector group of each utterance of the input matrix set, and the mixture weight and average vector of each normal distribution And a covariance matrix are obtained, and the obtained mixed normal distribution is output (step S2). In the next component content calculation unit 103, each normal distribution of the mixed normal distribution is regarded as one component.

  For fitting the mixed normal distribution (mixing weight, average vector, and covariance matrix estimation), for example, a general EM algorithm described in Reference 1 or the like is used. The mixture number M is an integer of 1 or more, and if it is increased, the acoustic feature quantity can be more precisely decomposed into components. However, since the number of parameters of the mixed normal distribution increases, the number of acoustic feature quantity vectors necessary for estimation is reduced. To increase. Usually, a mixing number M of about 512 is used.

  [Reference 1] C.M. Bishop, “Pattern recognition and machine learning,” pp.154-155, Springer Japan, 2008-07-01.

<Component content calculation unit 103>
Input: acoustic feature vector group (from acoustic feature extraction unit 101) and mixed normal distribution (from mixed normal distribution fitting unit 102) of each utterance in the matrix set
Output: Component content of each utterance in the mother set Processing: The component content calculation unit 103 uses the input acoustic feature vector group and mixed normal distribution of each utterance in the mother set as components of each utterance in the mother set. Calculate and output the content.

  There are a number M of mixed normal distribution components, and the content is calculated for each component. The component content of a certain utterance is the sum of the component contents of each acoustic feature vector of the utterance. The component content of one acoustic feature vector is calculated as follows.

  (1) The component content calculation unit 103 calculates the likelihood in all normal distributions from the first to the Mth for the target acoustic feature vector x. When the average vector of the mth normal distribution is μm and the covariance matrix is Sm, the likelihood Lm of the mth normal distribution for the acoustic feature vector x is calculated by the following equation. d is the number of dimensions of the acoustic feature vector.

  (2) The component content calculation unit 103 normalizes the obtained M likelihoods from L1 to LM so that the sum is 1.

  M normalized likelihoods from P1 to PM obtained in the procedure of (2) are the component contents of the acoustic feature vector x. The component content calculation unit 103 calculates the component content of each acoustic feature vector in the utterance, and calculates the component content of the utterance by taking the sum in the utterance for each component.

  The component content calculation unit 103 calculates the component content for each utterance in the matrix set according to the above procedure, and the component content of each utterance in the matrix set (how much each utterance contains each component). Is obtained (step S3).

<Speech selection unit 104>
Input: matrix set, component content of each utterance of the matrix set (from the component content calculation unit 103), mixed weight of each normal distribution of the mixed normal distribution (from the mixed normal distribution fitting unit 102), utterance upper limit C
Output: Learning set Processing: The utterance selection unit 104 selects and learns an utterance from the mother set using the component content of each utterance of the inputted mother set, the mixture weight of each normal distribution, and the utterance number upper limit C. Output as a set.

  Utterance selection is performed by the following procedure using the greedy method.

  (0) The utterance selection unit 104 initializes the learning set U to an empty set. Initialize the matrix set to a set whose elements are all utterances of the matrix set.

(1) The utterance selection unit 104 calculates an increase value of the diversity score when each utterance in the mother set is added to the learning set U. Rise value Improve _n diversity score when adding the n th utterance u _n in maternal set training set is calculated by the following equation (5) and (2).

w _i is the mixing weight of the i-th normal distribution, and f _iU is the sum of the i-th component contents of all utterances included in U.

  (2) The utterance selection unit 104 moves the utterance that increases the diversity score the largest from the mother set to the learning set U.

  (3) If the number of utterances in the learning set U is less than C, the utterance selection unit 104 returns to step (1) and repeats. The utterance selection unit 104 ends when the number of utterances in the learning set U reaches C, and outputs the learning set U as learning data.

  C is an integer of 1 or more and less than or equal to the number of utterances of the mother set, and represents the number of utterances of the learning set that is finally output. If C is increased, a learning set including more diverse speakers, recording devices, and ambient noises of the mother set can be obtained. However, since the number of utterances in the learning set increases, processing time and memory during learning of the speaker feature extraction model Increased usage. Usually, it is set to a value of about 3 to 50,000.

  Through procedures (1) and (2), utterances are sequentially selected so that the component ratio of each component in the learning set U approaches the mixture weight of the mixed normal distribution (= the component ratio of each component of the matrix set). . For this reason, an utterance set is selected so as to faithfully reproduce the component ratio of each component of the matrix set as much as possible within the constraint of the upper limit C of the number of utterances in the learning set, and is output as a learning set. .

  Since the diversity score D (U) is a submodular function, for example, when the function to be maximized is a submodular function, the same learning set as the above greedy method is used. You may use the speed-up method obtained with a small processing amount.

[Reference 2] Jure Leskovec, Andreas Krause, Carlos Guestrin, Christos Faloutsos, Jeanne VanBriesen and Natalie Glance, “Cost-effective outbreak detection in networks,” in Proceedings of the 13 ^th ACM SIGKDD international conference on Knowledge discovery and data mining, pp.420-429, 2007.

  With the above configuration, the utterance selection unit 104 makes the composition ratio of each component of the learning set close to the composition ratio of each component of the mother set (that is, various speakers, recording devices, and peripherals included in the mother set). A learning set in which an utterance is selected is output (so as to include noise uniformly) (step S4). By using the speaker feature extraction model learned from this learning set, speaker recognition can be performed with higher accuracy than when a learning set created by randomly selecting utterances is used.

[Program and recording medium]
The processes described in the learning data generating apparatus and method are not only executed in time series in the order described, but may also be executed in parallel or individually as required by the processing capability of the apparatus that executes the processes. Good.

  Further, when each process in the learning data selection device is realized by a computer, the processing contents of the functions that each apparatus should have are described by a program. Then, by executing this program on a computer, each process is realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used.

  Each processing means may be configured by executing a predetermined program on a computer, or at least a part of these processing contents may be realized by hardware.

  Needless to say, other modifications are possible without departing from the spirit of the present invention.

101 acoustic feature amount extraction unit 102 mixed normal distribution fitting unit 103 component content calculation unit 104 utterance selection unit

Claims (4)

An acoustic feature quantity extraction unit that extracts an acoustic feature quantity vector group from an audio signal of each utterance included in the matrix set;
A mixed normal distribution fitting unit that obtains a mixed normal distribution by applying a mixed normal distribution of a predetermined number of mixtures M to the acoustic feature quantity vector group;
A component content calculation unit that calculates the content of each component in the matrix set using the acoustic feature vector group, with each of the M normal distributions constituting the obtained mixed normal distribution as components,
Based on the content of each component in the matrix set, select the utterance from the utterances included in the matrix set so that the composition ratio of each component in the learning set is close to the composition ratio of each component in the matrix set An utterance selection unit that generates learning data by
A learning data generating apparatus including: The learning data generation device according to claim 1,
The utterance selection unit, based on the content of each component in the mother set, from among the utterances included in the mother set, the component ratio of each component in the learning set after adding the utterance is each in the mother set Generate learning data by repeating the process of selecting one utterance and adding it to the learning set so that it is close to the component composition ratio.
Learning data generation device. An acoustic feature extraction step for extracting an acoustic feature vector from a speech signal of each utterance included in the matrix set;
A mixed normal distribution fitting step for obtaining a mixed normal distribution by fitting a mixed normal distribution having a predetermined number of mixtures M to the acoustic feature vector group;
A component content calculation step for calculating the content of each component in the matrix set using the acoustic feature vector group, with each of the M normal distributions constituting the obtained mixed normal distribution as components,
Based on the content of each component in the matrix set, select the utterance from the utterances included in the matrix set so that the composition ratio of each component in the learning set is close to the composition ratio of each component in the matrix set An utterance selection step for generating learning data by
Learning data generation method including   The program for functioning a computer as each part of the learning data generation apparatus of Claim 1 or 2.

Images