JP2016156870A - Language identification model learning device, language identification device, language identification model learning method, language identification method, program, and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform highly accurate language identification.SOLUTION: A learning data storage part 1 stores a plurality of learning data items composed of groups of voice data by a plurality of languages combined with language labels representing languages of the respective voice data. A conversion model learning part 2 uses the learning data to learn a discrete symbol sequence conversion model outputting a discrete symbol sequence discretizing a posterior probability distribution of language labels with the voice data as input. A discrete symbol sequence conversion part 4 uses the discrete symbol sequence conversion model to convert the voice data of the learning data into a discrete symbol sequence. A language identification model learning part 5 uses the discrete symbol sequence of the learning data and the language labels to make the discrete signal sequence of the voice data input, and learns a language identification model outputting generation probability every language label.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a language identification technique for identifying in which language an input utterance is spoken.

  With the development of the global society, the need for allowing input of multiple languages is increasing in speech recognition technology. Therefore, there is a demand for advanced language identification technology for identifying which language of the input speech is (for example, English, Japanese, or Chinese).

  In the language identification technology, a framework is generally used in which the language likeness of each language is statistically grasped in advance to calculate and identify which language the input utterance is closest to. Specifically, a large number of pairs of speech data and language labels (information such as “This speech is spoken in Japanese”) is prepared, and a language classifier is constructed by grasping the language likeness of each language. .

  As a conventional language identification technique, there is a method described in Non-Patent Document 1. In Non-Patent Document 1, language identification is realized using a statistical model called a deep neural network. Specifically, using a neural network, the linguistic character is statistically captured in units of a few milliseconds. For example, when the possibility of language labels is 3 languages (Japanese, English, Chinese), the neural network learns a statistical model in units of frames of a certain voice width (for example, 25 milliseconds) A probability value representing which one of the above three languages corresponds when an arbitrary frame is input is calculated. The deep neural network learning method is a known technique.

  When performing language identification using a learned neural network, the input speech is divided into frames of the speech width used in the neural network learning for the input speech for which the language spoken is unknown, The probability value of which language is spoken for each frame is calculated. Thereafter, the probability values are averaged over all frames, and the language having the highest average value is identified. For example, if the input voice is 3 seconds and the length of one frame is defined as 25 milliseconds, there are 120 frames in the input voice. At this time, if the first frame of 120 frames is input to the learned deep neural network, for example, the probability value is “0.5 probability for English, 0.3 for Japanese, 0.2 for Chinese” Is output. After such processing is similarly obtained for all the remaining 119 frames, an average probability value is calculated for each language (for example, every English, every Japanese, every Chinese). That is, the probability values of all English (or Japanese and Chinese) from the first frame to the 120th frame are added and divided by 120 of the number of frames. As a result of such processing, for example, it is assumed that the average probability of English is 0.7, the average probability of Japanese is 0.1, and the average probability of Chinese is 0.2. In this case, the language classifier identifies that the language of the input speech is English, which is the language having the maximum probability.

  The reason for obtaining the probability value of which language is spoken by averaging the probability values of all frames is that there is not enough information to identify the language at each frame level. No matter how precise the neural network enables modeling, it is not possible to model enough to specify a language completely from speech in a few milliseconds. Specifically, when a probability value is calculated for each English speech for each frame, it is determined that the probability of English is high for the first frame, but the probability of English is low for the second frame. There are enough cases. In the conventional method, this problem is solved by averaging the whole, and language identification can be realized with a certain degree of accuracy.

Javier Gonzalez-Dominguez, Ignacio Lopez-Moreno, Pedro J. Moreno, Joaquin Gonzalez-Rodriguez, “Frame by Frame Language Identification in Short Utterances using Deep Neural Networks”, Neural Networks Special Issue: Neural Network Learning in Big Data, 2014.

  The framework of Non-Patent Document 1 solves the low discriminating ability at the frame level by averaging the probability values of all frames. However, if the whole is averaged, the change as a whole series cannot be grasped well. For example, when a certain English voice is applied to the language classifier, 3 frames out of 4 frames appear with a high probability of English, but 1 frame out of 4 frames may appear with a high probability of Japanese. . Even if such a phenomenon actually exists, if the whole is averaged and understood, it will be impossible to make use of the knowledge that “English speech is likely to cause such a phenomenon”.

  The fact that this problem leads to identification errors will be described with a more specific example. For example, the identification probability for each frame of an English speech is “P (English) = 0.5, P (Japanese) = 0.4, P (Chinese) = 0.1” in the first to third frames, and the fourth frame. Is assumed to be “P (English) = 0.1, P (Japanese) = 0.8, P (Chinese) = 0.1”. Here, P (*) represents a probability value that the frame is the language *. Taking this series as an average value, the probabilities are “P (English) = 0.4, P (Japanese) = 0.5, p (Chinese) = 0.1”. Since Japanese has a higher probability than English, in practice, even an English speech will cause an identification error that is identified as Japanese. As described above, it is often the case that the linguistic character cannot be grasped well at the frame level, and the accuracy for language identification is lacking simply by averaging.

  That is, the problem of the prior art described in Non-Patent Document 1 is that there is a lack of precise language identification due to the language-likeness being grasped by the information of the average value that does not include information relating to the variation of the language-likeness as a series. I can say that.

  In view of such a point, an object of the present invention is to realize language identification with higher accuracy using sequence information of language-likeness at a frame level.

  In order to solve the above-described problem, the language identification model learning device according to the first aspect of the present invention includes a plurality of pieces of learning data in which speech data in a plurality of languages and a language label representing the language of each speech data are paired. A learning data storage unit for storing, a conversion model learning unit for learning a discrete symbol sequence conversion model for outputting a discrete symbol sequence obtained by discretizing a posteriori probability distribution of a language label using speech data as input using the learning data; Using a discrete symbol sequence conversion model, a discrete symbol sequence converter that converts speech data of learning data into discrete symbol sequences, and a discrete symbol sequence of speech data using the discrete symbol sequences and language labels of learning data as inputs. A language identification model learning unit that learns a language identification model that outputs a generation probability for each language label.

  A language identification device according to a second aspect of the present invention includes a conversion model storage unit that stores a discrete symbol sequence conversion model generated by a language identification model learning device, and a language that stores a language identification model generated by a language identification model learning device. For each language label from the discrete symbol sequence of the input speech data using the identification model storage unit, the discrete symbol sequence conversion unit that converts the input speech data into a discrete symbol sequence using the discrete symbol sequence conversion model, and the language identification model And a language identification unit that outputs a language label that gives the maximum generation probability.

  The language identification technique of the present invention models frame level language-like sequence information captured by a neural network for each language, and performs language identification of input speech using the language identification model. Therefore, according to the present invention, it is possible to realize highly accurate language identification as compared with the case where language identification is performed with an average standard of language likeness.

FIG. 1 is a diagram illustrating a functional configuration of a language identification model learning device. FIG. 2 is a diagram illustrating a functional configuration of the conversion model learning unit. FIG. 3 is a diagram illustrating a functional configuration of the language identification model learning unit. FIG. 4 is a diagram illustrating a processing flow of the language identification model learning method. FIG. 5 is a diagram illustrating a functional configuration of the language identification device. FIG. 6 is a diagram illustrating a processing flow of the language identification method.

  Prior to the description of the embodiments, the basic concept of the present invention will be described.

  In the present invention, the language level of the frame level is discretized and captured. The language-specific information for each frame is, for example, “P (English) = 0.5, P (Japanese) = 0.4, P (Chinese) = 0.1”. It is very complicated to capture such a continuous expression as a sequence. On the other hand, if it is discretized information, it is easy to catch the series. Therefore, the language-like information for each frame is discretized, and the entire speech data is converted into a discrete symbol sequence. Information on the language likeness for each frame can be considered as a vector having a probability value for each language as an element. In other words, by dividing this vector space according to some criteria and determining the cluster number for each space, the language-like information can be converted into discrete symbols. For example, the language likeness in the first to third frames (for example, “P (English) = 0.5, P (Japanese) = 0.4, P (Chinese) = 0.1”) becomes cluster number 10, and the language likeness in the fourth frame. (For example, “P (English) = 0.1, P (Japanese) = 0.8, P (Chinese) = 0.1”) becomes cluster number 3, the discrete symbol sequence of this speech is “10,10,10 , 3 ”.

  The discrete symbol series is modeled for each language (in the above example, for each English, every Japanese, and every Chinese) using the existing symbol series modeling technology. A known technique called a language model can be used for modeling the symbol series. For example, if a language model called an N-gram model is used, the generation probability of N symbol sequences can be directly modeled.

  At the time of identification, the input speech data is converted into a discrete symbol sequence, the generation probability of each language is obtained from the discrete symbol sequence using the learned model, and the language that gives the maximum generation probability is identified.

  Hereinafter, embodiments of the present invention will be described in detail. In addition, the same number is attached | subjected to the component which has the same function in drawing, and duplication description is abbreviate | omitted.

  In the embodiment, a language identification model learning apparatus and method for learning a discrete symbol sequence conversion model and a language identification model using learning data, and language identification of speech data using the learned discrete symbol sequence conversion model and language identification model. A language identification apparatus and method to be performed will be described.

<Language identification model learning>
As shown in FIG. 1, the language identification model learning device of the embodiment includes a learning data storage unit 1, a conversion model learning unit 2, a conversion model storage unit 3, a discrete symbol sequence conversion unit 4, a language identification model learning unit 5, and The language identification model storage unit 6 is included, for example. The conversion model learning unit 2 includes, for example, a neural network learning unit 21 and a centroid generation unit 22 as shown in FIG. The language identification model learning unit 5 includes, for example, a learning data dividing unit 51 and a model learning unit 52 as shown in FIG.

  The language identification model learning device is configured, for example, by loading a special program into a known or dedicated computer having a central processing unit (CPU), a main memory (RAM), and the like. It is a special device. For example, the language identification model learning device executes each process under the control of the central processing unit. The data input to the language identification model learning device and the data obtained in each process are stored in, for example, the main storage device, and the data stored in the main storage device is read out as needed for other processing. Used. In addition, at least a part of each processing unit of the language identification model learning device may be configured by hardware such as an integrated circuit.

  Each storage unit included in the language identification model learning device includes, for example, a main storage device such as a RAM (Random Access Memory), an auxiliary storage device configured by a semiconductor memory element such as a hard disk, an optical disk, or a flash memory (Flash Memory), Alternatively, it can be configured by middleware such as a relational database or key-value store. Each storage unit included in the language model creation device is only required to be logically divided, and may be stored in one physical storage device.

  The learning data storage unit 1 stores K (≧ 2) pieces of learning data. The learning data is a set of speech data that includes utterances spoken in T (≧ 2) languages and a language label that indicates in which language each speech data is spoken.

  With reference to FIG. 4, a processing procedure of the language identification model learning method of the embodiment will be described.

  In steps S11 to S12, the conversion model learning unit 2 uses the K pieces of learning data stored in the learning data storage unit 1 to learn a discrete symbol sequence conversion model that converts speech data into a discrete symbol sequence. The discrete symbol sequence conversion model is stored in the conversion model storage unit 3. The discrete symbol sequence is a sequence obtained by discretizing the posterior probability distribution representing the language-likeness of each frame. The discrete symbol sequence conversion model includes a neural network for obtaining a posterior probability distribution from speech data for each frame and a centroid of each cluster when the posterior probability distribution is clustered. The conversion model learning unit 2 learns a frame-level neural network from learning data, and converts each learning data into a series of posterior probability distributions using the learning data, as in the prior art. Thereafter, in order to efficiently discretize all frames of all data, the posterior probability distribution of the learning data is classified into M (≧ 2) clusters by K-means clustering, and the centroid of each cluster is learned. Hereinafter, the process of the conversion model learning unit 2 will be described in more detail.

  In step S <b> 11, the neural network learning unit 21 learns a neural network from the K pieces of learning data stored in the learning data storage unit 1. Similar to that used in the prior art, this neural network receives a feature amount for speech data of a frame and outputs a posterior probability distribution indicating which language the speech of the speech data is from. The frame length is several milliseconds, for example, 25 milliseconds as in the prior art. An arbitrary feature amount such as a known Mel Frequency Cepstral Coefficient (MFCC) can be used as the feature amount for the audio data of the frame.

  In this neural network, the value of the number T of language labels included in the learning data is the size of the output layer. For example, if there are three types of language labels assigned to K audio data, “Japanese”, “English”, and “Chinese”, T = 3 and the size of the output layer is 3. The value of the total number K of learning data is K = 30,000, assuming that there are 10,000 pieces of speech data for each language in the case of 3-language identification. The number and shape of the layers of the neural network can take any form. For example, a neural network in which the number of intermediate layers is 8 and the number of nodes in each layer is 2,048 can be used. For the neural network learning method, refer to Non-Patent Document 1 above.

  In step S12, the centroid generation unit 22 uses the neural network learned by the neural network learning unit 21 to convert each frame of the K learning data into a posterior probability distribution, and generates a sequence of K posterior probability distributions. obtain. Each posterior probability distribution is obtained as a three-dimensional vector if the above three-language identification is performed. Subsequently, the centroid generation unit 22 performs clustering in order to convert the posterior probability distribution (vector) into discrete symbols. The number of clusters M at the time of clustering is given manually. For example, M gives 64 or 128. As the clustering method, known K-means clustering can be used. When K-means clustering is performed, a centroid vector of each cluster can be obtained. The centroid is expressed as a vector in the same way as the posterior probability distribution. Each centroid is given an index (identification symbol). For example, an identification symbol “1” is given to the first centroid, and an identification symbol “2” is given to the second centroid.

  In step S13, the discrete symbol sequence conversion unit 4 converts the speech data of the learning data into a discrete symbol sequence using the discrete symbol sequence conversion model learned by the conversion model learning unit 2. Specifically, using a neural network, each frame of K learning data is converted to a posterior probability distribution (vector), and the Euclidean distance between each posterior probability distribution and each of the M centroids is measured for each frame. Convert to the nearest cluster identifier. For example, when the number of frames of a certain speech is 5, the posterior probability distribution is obtained for each of the 5 frames and is discretized by the above-described method. For example, the first frame is converted to the identification symbol “1”, the second to third frames are converted to the identification symbol “5”, the fourth frame is converted to the identification symbol “2”, and the fifth frame is converted to the identification symbol “3”. As a result, it is converted into a discrete symbol sequence of “1, 5, 5, 2, 3”. This is performed for all K learning data, and all the learning data is converted into a discrete symbol sequence.

  In steps S14 to S15, the language identification model learning unit 5 uses the discrete symbol sequence converted from the learning data by the discrete symbol sequence conversion unit 4 and the language label of the learning data stored in the learning data storage unit 1. Learn the language identification model. The language identification model is stored in the language identification model storage unit 6. The language identification model is composed of T discrete symbol sequence models learned for each language label. Hereinafter, the processing procedure of the language identification model learning unit 5 will be described in more detail.

  In step S14, the data dividing unit 51 divides the discrete symbol sequence of the learning data for each language label. Since the K learning data have language labels, they are divided into sets each having the same language label. That is, the same number of learning data sets as the number T of language label types can be created. For example, if there are three types of language labels, “Japanese”, “English”, and “Chinese”, they can be divided into a set of three learning data.

  In step S15, the model learning unit 52 learns the discrete symbol sequence model for each language label using the learning data divided for each language label, and aggregates the discrete symbol sequence models for each language and outputs a language identification model. To do. A technique called a language model can be used for modeling discrete symbol sequences. A discrete symbol sequence can be modeled using an arbitrary language model. For example, a known language model called an N-gram model can be used. The N-gram model can directly model the generation probability of N discrete symbol sequences. When learning an N-gram model, for example, “the probability that a discrete symbol sequence continues with“ 1,2 ”followed by a“ 3 ”is 0.3, and the probability that“ 1 ”continues is 0.2” is obtained. be able to. Then, by using the learned N-gram model, the generation probability can be obtained for an arbitrary discrete symbol sequence. The model learning unit 52 aggregates T discrete symbol sequence models learned for each language to generate a language identification model, and stores the language identification model in the language identification model storage unit 6.

<Language identification>
As illustrated in FIG. 5, the language identification device according to the embodiment includes a conversion model storage unit 3, a discrete symbol sequence conversion unit 4, a language identification model storage unit 6, and a language identification unit 7, for example. The conversion model storage unit 3, the discrete symbol sequence conversion unit 4, and the language identification model storage unit 6 are the same as the components included in the language identification model learning device. The conversion model storage unit 3 stores a discrete symbol sequence conversion model generated by the language identification model learning device. The language identification model storage unit 6 stores a language identification model generated by a language identification model learning device.

  The language identification device is, for example, a special program configured by reading a special program into a known or dedicated computer having a central processing unit (CPU), a main storage device (RAM), and the like. Device. For example, the language identification device executes each process under the control of the central processing unit. The data input to the language identification device and the data obtained in each process are stored in the main storage device, for example, and the data stored in the main storage device is read out as needed and used for other processing. The Further, at least a part of each processing unit of the language identification device may be configured by hardware such as an integrated circuit.

  With reference to FIG. 6, a processing procedure of the language identification method of the embodiment will be described.

  In step S21, the discrete symbol sequence conversion unit 4 converts the input speech data into a discrete symbol sequence using the discrete symbol sequence conversion model stored in the conversion model storage unit 3. The process of the discrete symbol sequence conversion unit 4 is the same as that in step S13 described above.

  In step S22, the language identification unit 7 uses the language identification model stored in the language identification model storage unit 6 and the input speech data from the discrete symbol sequence of the input speech data obtained by the discrete symbol sequence conversion unit 4. Find the language label that represents the language. More specifically, a probability value of language likelihood is calculated for each language label using a discrete symbol sequence model for each language label constituting the language identification model. Since the discrete symbol sequence model of each language label is represented as a language model, the generation probability can be obtained for an arbitrary symbol sequence as described above. Therefore, it is possible to calculate the generation probability of a language label for input speech data converted into a discrete symbol sequence. Language identification can be realized by obtaining a generation probability for each language label and then outputting a language label giving the maximum probability value. For example, if there is a discrete symbol series model of `` Japanese '' `` English '' `` Chinese '', the generation probability of each language is `` P (Japanese) = 0.05, P (English) = 0.02, P (Chinese) = If it is 0.0001, the language label giving the maximum is “Japanese”, so “Japanese” is output as the language label.

  By configuring the language identification technique of the present invention as described above, it is possible to realize language identification that makes full use of the frame level language-like sequence information captured by the neural network. Thereby, the performance of language identification can be improved as compared with the prior art in which language identification is performed with an average standard of language likeness.

  The present invention is not limited to the above-described embodiment, and it goes without saying that modifications can be made as appropriate without departing from the spirit of the present invention. The various processes described in the above embodiment may be executed not only in time series according to the order of description, but also in parallel or individually as required by the processing capability of the apparatus that executes the processes or as necessary.

[Program, recording medium]
When various processing functions in each device described in the above embodiment are realized by a computer, the processing contents of the functions that each device should have are described by a program. Then, by executing this program on a computer, various processing functions in each of the above devices are realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used.

  This program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. When executing the process, the computer reads a program stored in its own recording medium and executes a process according to the read program. As another execution form of the program, the computer may directly read the program from a portable recording medium and execute processing according to the program, and the program is transferred from the server computer to the computer. Each time, the processing according to the received program may be executed sequentially. A configuration in which the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes a processing function only by an execution instruction and result acquisition without transferring a program from the server computer to the computer. It is good. Note that the program in this embodiment includes information that is used for processing by an electronic computer and that conforms to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer).

  In this embodiment, the present apparatus is configured by executing a predetermined program on a computer. However, at least a part of these processing contents may be realized by hardware.

1 learning data storage unit 2 conversion model learning unit 21 neural network learning unit 22 centroid generation unit 3 conversion model storage unit 4 discrete symbol sequence conversion unit 5 language identification model learning unit 51 learning data division unit 52 model learning unit 6 language identification model Memory

Claims (7)

A learning data storage unit for storing a plurality of learning data in which voice data in a plurality of languages and a language label representing the language of each voice data are paired;
A conversion model learning unit that learns a discrete symbol sequence conversion model that outputs a discrete symbol sequence obtained by discretizing the posterior probability distribution of the language label using the learning data as input.
Using the discrete symbol sequence conversion model, a discrete symbol sequence conversion unit that converts speech data of the learning data into the discrete symbol sequence;
A language identification model learning unit that learns a language identification model that uses a discrete symbol sequence of speech data as input and outputs a generation probability for each language label using the discrete symbol sequence and language label of the learning data;
A language identification model learning device including: The language identification model learning device according to claim 1,
The conversion model learning unit
A neural network learning unit that learns a neural network that uses the learning data as input and outputs a posterior probability distribution of the language label, using a feature amount in units of frames of speech data as input,
Clustering the posterior probability distribution generated from the learning data using the neural network, and generating a centroid for each cluster;
Including
The discrete symbol sequence conversion unit obtains the posterior probability distribution of the language label from the feature amount of the learning data in frame units, and discretizes the posterior probability distribution based on the distance between the posterior probability distribution and the centroid. Is what
The language identification model learning unit
A learning data dividing unit that divides the learning data for each language label to generate learning data by language label;
Using the discrete symbol series and language labels of the learning data for each language label, learning a discrete symbol series model for each language label that determines the generation probability of the language label from the discrete symbol series of speech data, and the above discrete symbol series model for each language label A model learning unit that generates the language identification model by aggregating
A language identification model learning device including: A conversion model storage unit storing a discrete symbol sequence conversion model generated by the language identification model learning device according to claim 1;
A language identification model storage unit storing a language identification model generated by the language identification model learning device according to claim 1;
A discrete symbol sequence conversion unit that converts input speech data into a discrete symbol sequence using the discrete symbol sequence conversion model;
Using the language identification model, a language identification unit that obtains a generation probability for each language label from a discrete symbol sequence of the input speech data and outputs a language label that gives the maximum generation probability;
Language identification device including The learning data storage unit stores a plurality of learning data in which voice data in a plurality of languages and a language label representing the language of each voice data are paired,
A conversion model learning step for learning a discrete symbol sequence conversion model for outputting a discrete symbol sequence obtained by discretizing the posterior probability distribution of the language label, using the learning data as input, using the learning data;
A discrete symbol sequence conversion unit that converts the speech data of the learning data into the discrete symbol sequence using the discrete symbol sequence conversion model;
A language identification model learning unit learns a language identification model that receives a discrete symbol sequence of speech data and outputs a generation probability for each language label, using the discrete symbol sequence and language label of the learning data. Learning steps,
A language identification model learning method including A discrete symbol sequence conversion model generated by the language identification model learning method according to claim 4 is stored in the conversion model storage unit,
A language identification model generated by the language identification model learning method according to claim 4 is stored in the language identification model storage unit,
A discrete symbol sequence conversion unit that converts the input speech data into a discrete symbol sequence using the discrete symbol sequence conversion model;
A language identification step for obtaining a generation probability for each language label from a discrete symbol sequence of the input speech data using the language identification model, and outputting a language label that gives the maximum generation probability;
Language identification method.   A program for causing a computer to function as the language identification model learning device according to claim 1 or 2 or the language identification device according to claim 3.   A computer-readable recording medium on which a program for causing a computer to function as the language identification model learning device according to claim 1 or 2 or the language identification device according to claim 3 is recorded.

Images