JP5860439B2 - Language model creation device and method, program and recording medium - Google Patents

Description

  The present invention relates to a language model creation apparatus and method, a program, and a recording medium for constructing a language model that achieves high speech recognition performance by efficiently using language resources other than speech transcription related to a target task.

  Speech recognition requires a language model for linguistic prediction. As the language model, an N-gram language model that is very compatible with speech recognition decoding is generally used. The N-gram language model can be easily learned if there is a learning text, and various learning methods have been proposed so far. For example, the learning method of the N-gram language model is described in Non-Patent Document 1 and is well known.

  As a method for realizing highly accurate speech recognition, it is known that it is effective to use a language model specialized for a speech task (target task) to be recognized. Specialization means giving a high probability of occurrence to the language phenomenon often used in the task. That is, when recognizing news speech, a language model specialized for news speech is effective, and when recognizing call center speech, a language model specialized for call center speech is effective. In general, a language model specialized for a specific task is learned by using a text (transcribed text) obtained by manually transcribing the voice of the task.

  However, collecting a large amount of human transcription is accompanied by an increase in costs such as time and labor costs. Therefore, a technique for constructing a language model by using another language resource relating to the task without using the transcription of the target task is used. This technique is called language model task adaptation and is described in Non-Patent Documents 2 and 3.

  Language model task adaptation is a method of constructing a language model by constructing each N-gram language model from a plurality of external language resources and linearly interpolating the respective probabilities. Although this technique is simple, it is still widely used.

Kitakenji "Language and Computation-4 Stochastic Language Model" The University of Tokyo Press, pp.57-62. Kiyohiro Shikano et al. “IT Text Speech Recognition System” Ohm Publishing Office R. Iyer, and M. Ostendorf, “Modeling long distance dependence in language: topic mixture vs dynamic cache models”, IEEE Transactions on Speech and Audio Processing, vol.7, no.1, pp. 30-36, 1996.

However, in spoken language tasks such as contact centers, text with the same quality as transcripts is rarely obtained using language model task adaptation.
This is because language information such as the Web has almost no spoken language text information. For example, in the case where the communication voice of a communication contact center is the target task, the nouns used are similar if the communication Web text is used as an external language resource, but the style is written. Due to this difference in style, there is a problem that the transcription text that matches the task of spoken language cannot be obtained.

  The present invention has been made in view of this problem, and language model creation that flexibly adapts a language model task from an external language resource that does not perfectly match the task and constructs a high-performance language model for the task. An object is to provide an apparatus, a method thereof, a program thereof, and a recording medium.

  The language model creation apparatus of the present invention includes an LMLM learning unit, an LMLM task adaptation unit, a pseudo learning text creation unit, and an N-gram language model creation unit. The LMLM learning unit receives a plurality of learning texts, generates a plurality of hidden sequences that are hidden word sequences respectively corresponding to the word strings of the respective learning texts, and uses latent word-latent word probabilities from the hidden sequences. A latent word language model composed of two probability distributions of a certain sentence pattern model and a vocabulary model of latent word-observed word probability is learned for each of the plurality of learning texts. The LMLM task adaptation unit receives a plurality of latent language models, separates the latent language model into a sentence model and a vocabulary model, and sets a combination of a plurality of sentence models and vocabulary models as weighted tasks. Generate as a language model. The pseudo learning text creation unit receives the task adaptive latent word language model as an input, generates a latent word series, and generates a pseudo learning text from the latent word series. The N-gram language model creation unit creates pseudo-text as an input and counts the frequency of all N words in the pseudo-text to create an LMLM-like N-gram language model.

  According to the language model creating apparatus of the present invention, a latent language model is used to separate a language resource into two components, a sentence-type aspect and a lexical aspect, and a sentence-type aspect and a lexical aspect. Are appropriately mixed between language resources to perform language model task adaptation, and an N-gram language model is constructed for the task. Therefore, a high-performance language model adapted to the task can be constructed from external language resources having information useful only for the sentence-type aspect and external language resources having information useful only for the lexical aspect.

The figure which shows the function structural example of the language model creation apparatus 100 of this invention. The figure which shows the operation | movement flow of the language model production apparatus 100. The figure for demonstrating a latent word series and an observation word series. The figure which shows the function structural example of the LMLM learning part 110. FIG. The figure which shows the operation | movement flow of the LMLM learning part 110. The figure which shows the function structural example of the LMLM task adaptation part. The figure which shows the operation | movement flow of the LMLM task adaptation part. The figure which shows the function structural example of the pseudo learning text preparation part 140. FIG. The figure which shows the operation | movement flow of the pseudo learning text preparation part 140. FIG. The figure which shows the function structural example of the language model creation apparatus 200 of this invention. The figure which shows the operation | movement flow of the language model production apparatus 200.

  Embodiments of the present invention will be described below with reference to the drawings. The same reference numerals are given to the same components in a plurality of drawings, and the description will not be repeated.

  FIG. 1 shows a functional configuration example of the language model creation device 100 of the present invention. The operation flow is shown in FIG. The language model creation device 100 includes an LMLM learning unit 110, an LMLM task adaptation unit 130, a pseudo learning text creation unit 140, an N-gram language model creation unit 160, and a control unit 180. LVLM (Latent Words Language Model) is a latent language model. The language model creation apparatus 100 is realized by a predetermined program being read into a computer composed of, for example, a ROM, a RAM, and a CPU, and the CPU executing the program. The same applies to other embodiments described below.

  The LMLM learning unit 110 receives a plurality of learning texts, generates a plurality of hidden sequences that are hidden word sequences respectively corresponding to the word strings of the respective learning texts, and latent word-latent word probabilities from the hidden sequences. A latent word language model comprising two probability distributions of a sentence pattern model and a vocabulary model of latent word-observed word probability is learned for each of the plurality of learning texts (step S110). The learning of one LMLM is repeated for the number of hidden sequences (step S1801). The control unit 180 controls this repetitive operation. A plurality of learned latent language models having the same number as the hidden series are recorded as a set of models, and constitute a latent language model 120. A latent word corresponds to a class in the class N-gram language model, and represents a representative word when words having similar meanings and syntactic roles are grouped in a certain context.

  FIG. 3 illustrates the relationship between latent words and observed words. The observation word is a word appearing on the text, for example, a word string (observation word string) constituting a sentence of the learning text “Today is good weather” as shown in the figure. For this observed word, the latent word is expressed using a representative word similar to the observed word. The observation word “today” corresponds to the latent word “tomorrow” and is represented as a representative of a plurality of similar words such as “tomorrow”, “yesterday”, and “today”.

The learning text is a language resource and is divided into words. The latent word language model has _two probability distributions of a latent word-latent word probability distribution P (h _k | h _k−2 , h _k−1 ) and a latent word-observed word probability distribution P (w _k | h _k ). Consists of. The latent word h _k is a latent variable in the latent word language model, and the observed word w _k represents a word that actually appears in the text.

  The hidden sequence is, for example, that the observed word sequence is “apple mandarin pine” and if the number of words is assumed to be only those four words, one of the hidden sequences is, for example, one sentence of “apple apple apple”. This hidden sequence is generated in a predetermined number (plural).

  The LMLM task adaptation unit 130 receives a plurality of latent language models learned by the LMLM learning unit 110, separates the latent language model into a sentence model and a vocabulary model, and weights and mixes the plurality of sentence models and the vocabulary models respectively. The set is generated as a task adaptive latent language model (step S130). The LMLM task adaptation unit 130 constructs a single latent language model using N latent language models learned from N language resources. Specifically, the sentence model of the latent language model is mixed between the sentence model, and the vocabulary model is mixed between the vocabulary models.

  The pseudo learning text creation unit 140 receives the task adaptation latent word language model generated by the LMLM task adaptation unit 130 as an input, generates a latent word sequence, and generates a pseudo learning text from the latent word sequence (step S140). This step S140 is repeated until M observation words are generated (No in step S1802), and the pseudo learning text 150 is created. M may be given from the outside, or may be set as a constant in the pseudo learning text creation unit 140 in advance.

The N-gram language model creation unit 160 receives the pseudo-learning text 150 as an input and counts the frequency of all N words in the pseudo-learning text to create an LMLM-like N-gram language model (step S160). . In the case of speech recognition, N is generally often N = 3. For example, an LMLM-like language model having a 3-gram structure gives a probability distribution of P _lwlm (w _k | w _k−2 , w _k−1 ). The learning method of the N-gram language model is well known (Non-Patent Document 1). The created N-gram language model is generally recorded as a set of models, and constitutes an LMLM-like N-gram language model 170.

  As described above, according to the language model creation device 100, an external language resource having information useful only for the sentence-type aspect or an external language resource having information useful only for the lexical aspect is adapted to the task. An LVLM-like N-gram language model can be created. Hereinafter, the operation of the language model creating apparatus 100 will be described in more detail by showing more specific functional configuration examples of the respective units.

[LVLM Learning Department]
The learning of the latent language model is a problem of estimating the allocation of latent words for each word of the input learning text. That is, if there is a learning text (sequence of observation words) of “w ₁ , w ₂ ,..., W _L ” with the total number of words _L , each observation word of “w ₁ ” “w ₂ ”. It can be said that the latent words “h ₁ ” “h ₂ ”... “H _L ” are estimated. If this allocation can be estimated, P (h _k | h _k−2 , h _k−1 ) is constructed by learning an N-gram language model for the latent word sequence “h ₁ , h ₂ ,..., H _L ”. P (w _k | h _k ) can be constructed by learning a unigram language model for “h ₁ → w ₁ ”, “h ₂ → w ₂ ”... “H _L → w _L ”.

  FIG. 4 shows a more specific functional configuration example of the LMLM learning unit 110. The operation flow is shown in FIG. The LMLM learning unit 110 includes hidden sequence estimation means 1101 and probability generation means 1102.

  The hidden sequence estimation unit 1101 receives the learning text as input, and generates hidden word sequences corresponding to the word strings of the learning text as hidden sequences by the number of information B (B ≧ 2) (No in step S1801a). (Step S1101). The information B is a predetermined number and may be given from the outside. If the set of words included in the learning text is W, any word included in W is assigned to each latent word.

As in the example above, if the learning text is a sentence consisting of four words “I am apple apple pine,” W is four words “apple,” “mandarin,” “pine,” and “is.” . In this example, “apple apple apple is”, “mandarin orange apple is”, “pine pine pine”, etc. are candidates for the hidden series. Hidden series in this case, likely number of ^{4 four} (W = 4). The assignment of latent words constituting this hidden sequence is estimated by a known method called Gibbs sampling. 4 ^four hidden sequence of this embodiment can be said to represent the sentence pattern.

The probability generation means 1102 generates a latent word-latent word probability P (h _k | h _k-2 , h _k-1 ) and a latent word-observed word probability P (w _k | h _k ) from the hidden sequence (step) S1102a).

  For example, suppose that “I am an apple apple apple” is selected as the hidden series. If the hidden sequence is determined, the latent word-latent word probability P (apple | apple, apple) can be obtained by, for example, Equation (1) in the case of maximum likelihood estimation.

  When “apple apple apple” is a hidden sequence, the number of occurrences of “apple apple” in the hidden sequence is 2 and the number of occurrences of “apple apple apple” is 1. Therefore, the latent word-latent probability P (apple) | Apple, apple) = 1/2 The latent word-latent word probability P (apple | apple, apple) represents the probability that the fruit name often appears after the fruit name with the fruit name “apple” as a representative word. This latent word-latent word probability P represents a sentence model. In addition to the value obtained by maximum likelihood estimation, a probability estimated by an arbitrary method such as back-off smoothing can be used. Backoff smoothing is a known technique.

Next, the probability generation means 1102 calculates the latent word-observed word probability P (w _k | h _k ) (
Step S1102b). The latent word-observed word probability P (mandarin | apple) can be obtained by Equation (3) when the learning text is “It is apple orange pine”, for example, in the case of maximum likelihood estimation.

  The number of latent word “Ringo” in the hidden sequence “Ringo-Ringo-Ringo” is 3, and the number of times the observed word in the learning text is “Mikan” is 1. = 1/3. The latent word-observed word probability P (mandarin | apple) represents the probability that "mandarin orange" appears as an observed word. This latent word-observed word probability P represents a vocabulary model. As described above, the value estimated by a method other than the maximum likelihood estimation (for example, back-off smoothing) can be used for the probability.

As described above, the probability generation unit 1102 uses the latent word-latent word probability P (h _k | h _k-2 , h _k-1 ) and the latent word-observed word probability P (w _k | h _k ) for the hidden sequence. ). This set is expressed as a latent language model. The latent word-latent word probability P (h _k | h _k-2 , h _k-1 ) and the latent word-observed word probability P (w _k | h _k ) are repeated until the number of information B is calculated. (Step S1801b). The value of the information B is set to be large when the latent language model is desired to be detailed. If the value of the information B is about 10, it can be said that it is somewhat detailed.

The LMLM learning unit 110 includes the latent word-latent word probability P _n (h _k | h _k-2 , h _k-1 ) and the latent word-observed word of the number of learning texts to which the above processing is input. A latent post-language model consisting of a set (n: 1, 2,... N) of probabilities P _n (w _k | h _k ) is learned. That is, N latent language models are generated from the N learning texts.

[LVLM task adaptation part]
FIG. 6 shows a more specific functional configuration example of the LMLM task adaptation unit 130. The operation flow is shown in FIG. The LMLM task adaptation unit 130 includes a model division unit 1301, a sentence pattern model adaptation unit 1302, and a vocabulary model adaptation unit 1303.

The model dividing unit 1301 divides the plurality of latent language models learned by the LMLM learning unit 110 into a sentence model and a vocabulary model (step S1301). The first latent language model is divided into a sentence pattern model P ₁ (h _k | h _k−2 , h _k−1 ) and a vocabulary model P ₁ (w _k | h _k ). This model division step is repeated by the number of latent language models (N) (step S1802a).

  The sentence pattern model adaptation unit 1302 generates a task adaptation sentence pattern model 1304 in which the N sentence pattern models divided by the model dividing unit 1301 are weighted and mixed (step S1302). The task adaptive sentence model can be expressed by equation (4).

Here, ω _n is a weight for the n-th sentence pattern model. This weight is determined based on the constraint shown in the following equation.

The weight ω _n may be given in advance to each of a plurality of inputs to the LMLM learning unit 110 depending on the degree of whether or not the learning text is suitable for sentence pattern model extraction. For example, when there are three learning texts, the weights are determined in advance as ω ₁ = 0.5, ω ₂ = 0.4, ω ₃ = 0.1. ω ₁ = 0.5 means that in this example, the learning text is most likely to use the sentence pattern. For example, if the target domain is a communication contact center and the external language resource is a learning text transcribed from a voice recording a conversation between an operator and a user, the sentence pattern is considered to be reliable. Alternatively, the language model creation device 100 is configured to ask the user who uses the language model creation device 100 how many weights ω _n should be used when the LMLM task adaptation unit 130 processes the latent language model. May be.

  The vocabulary model adaptation unit 1303 generates a task adaptation vocabulary model 1305 in which the N vocabulary models divided by the model division unit 1301 are weighted and mixed (step S1303). The sentence pattern model adaptation step (step S1302) and the vocabulary model adaptation step (step S1303) are repeated by the number (N) of latent language models (step S1802b). The task adaptive vocabulary model can be expressed by equation (6).

Here, μ _n is a weight for the nth vocabulary model. This weight is determined based on the constraint shown in the following equation.

The weight μ _n for the vocabulary model is a value determined in advance according to the degree of whether or not the learning text is suitable for extraction of the vocabulary model. For example, if the target domain is a communication contact center and the external language resource is a communication-related news article on the Web, the vocabulary contained therein is considered to be trustworthy and is therefore determined to be a large value. The method of determining the weight μ _n is the same as the weight ω _n .

[Pseudo-learning text creation part]
FIG. 8 shows a more specific functional configuration example of the pseudo learning text creation unit 140. The operation flow is shown in FIG. Pseudo-learning text creating unit 140, the task adaptation pseudo-learning text from LWLM _{"w 1, w 2, ...,} w M " is intended to generate M word.

The pseudo-learning text creation unit 140 includes latent word generation means 1401 and observation word generation means 1402. The latent word generation means 1401 receives the task adaptive sentence pattern model P _all (h _k | h _k−2 , h _k−1 ) and the latent word history, and randomly generates the latent word h _k (step S1401).

Since the latent word history does not exist at first, the latent word generating unit 1401 randomly generates the initial latent word h _{1 based} on the task adaptive sentence pattern model P _all (h ₁ | −, −) (step S1401a). The latent word h ₁ is generated based on a well-known SampleOne algorithm that samples one word randomly from a certain discrete probability distribution.

The latent word generating means 1401 generates a uniform random number (rand) (step S1401b), and determines the first latent word h ₁ according to the referenced latent word-latent word probability P _all (h ₁ | −, −) (step S1401b). S1401c). The latent word h ₁ is determined by the relationship between the uniform random number (rand) and the probability of the latent word h ₁ . For example, P ₁ (apple |-,-) = 0.3, P ₁ (mandarin |-,-) = 0.3, P ₁ (pine |-,-) = 0.3, P ₁ (is |- ,-) = 0.1. When rand = 0.1, the first latent word h ₁ = “apple” is determined. When Rand = 0.7, the latent word h ₁ = “pine”, and when rand = 0.95, the latent word h ₁ = “is”. The process of determining the potential word h ₁ is a SampleOne algorithm.

The observed word generation means 1402 receives the latent word h ₁ as input, refers to the task adaptive vocabulary model P _all (w _k | h _k ), and determines the observed word based on the relationship between the latent word-observed word probability and the uniform random number. and, pseudo- the determined observed words and outputs to the outside as a learning text, and outputs a potential word history corresponding to the output were observed word to potential word generation unit 1401 (step S1402).

When the latent word h ₁ is input, the observed word generation unit 1402 utters a uniform random number (rand) (step S1402b), and the uniform random number (rand) and the latent word-observed word probability P _all (w ₁ | h to determine the observed word _{w 1} in relation to ₁₎ (step S1402c). With the determined observation words w ₁ is output as the learning text pseudo- (Step S1402d), and outputs a potential word history h ₁ corresponding to the monitoring words w ₁ output to the potential language model selecting means 1401 (step S1402e).

For example, the observation word w ₁ has the latent word-observation word probability P _all (w ₁ | h ₁ ) observation word probabilities of “apple” = 0.3, “mandarin orange” = 0.3, “pine” = When 0.3, “is” = 0.1, when the random number is rand = 0.1, “apple”, when rand = 0.4, “mandarin”, rand = 0.7 The time is determined as “Pine”, and when rand = 0.95, it is “I”.

When outputting the observation word _{w 1,} pseudo learning text creation section 140 is repeated the processing up to step S1401a~ Step S1402e described above, until the M observations word _{w M} is outputted (No in step S1802). The value of M can be a pseudo-learning text represents well the nature of the potential language model larger. M may be given from the outside, or may be set in the observation word generation unit 1402 in advance.

  FIG. 10 shows a functional configuration example of the language model creation device 200 of the present invention. The operation flow is shown in FIG. The language model creation device 200 creates a mixed N-gram language model in which a conventional N-gram language model and the language model created by the language model creation device 100 described above are mixed at a predetermined ratio.

The language model learning apparatus 200 includes an RWLM-like language model creation unit 210, N language model creation units 220 _{1 to} 220 _N , an LMLM-like N-gram language model 230, and N N-gram language models 240 _1. -240 _N , the mixed N-gram language model creation part 250, and the control part 260 are comprised. The LMLM-like language model creation unit 210 is the same as the N-gram language model creation unit 160 constituting the language model creation device 100 described above. For convenience of explanation, it is referred to as an LVLM-like language model creation unit 210, which is the same as the N-gram language model 240 _n (n: 1 to N).

LWLM language model generating unit 210 is input with pseudo training text 150 to pseudo-learning text creation section 140 of the language model creating apparatus 100 has generated, the frequency of words in all N tuples in the pseudo training text And an N-gram language model is created as an LVLM-like language model P _lwlm (w _k | w _k−2 , w _k−1 ) (step S210).

Learning texts 1 to N are respectively input to the _N language model creation units 220 _{1 to} 220 _N , and each language model creation unit 220 _n (n: 1 to N) includes all N sets in each learning text. A plurality of N-gram language models 240 ₁ to 240 _N are created by counting the frequency of the words (step S220). The N-gram language models 240 ₁ to 240 _N are P ₁ (w _k | w _k−2 , w _k−1 ) to P _N (w _k | w _k−2 , w _k−1 ).

The mixed N-gram language model creation unit 250 includes an LMLM-like language model P _lwlm (w _k | w _k−2 , w _k−1 ) and a plurality of N-gram language models P ₁ (w _k | w _k−2 , Using w _k−1 ) to P _N (w _k | w _k−2 , w _k−1 ) as input, a single mixed N-gram language model is created by mixing all language models (step S250). Mixing follows the formula: The mixing of language models (step S250) is repeated until all N-gram language models are completed (No in step S260).

Here, λ _n and λ _lwlm are weights for each language model and satisfy the following constraints.

The values of λ _n and λ _lwlm are determined in advance. For example, if it is desired to increase the weight of the LMLM-like language model P _lwlm (w _k | w _k−2 , w _k−1 ), λ _lwlm is set large, and if it is _desired to be small, λ _lwlm is set small. The mixed N-gram language model thus mixed can be a language model that complements the characteristics of the latent language model and the characteristics of the conventional N-gram language model.

[Evaluation experiment]
For the purpose of evaluating the performance of the language model created by the language model creation device of the present invention, an evaluation experiment was performed to compare speech recognition error rates. As a domain to be evaluated, a contact center simulated call (contact center: A) was used. One hour of the simulated call was transcribed as learning data, and the other two hours were used as test data. In addition, four types of external language resources were used. Specifically, Twitter data, blog data, Japanese spoken corpus (CSJ), and other domain contact center voice transcripts (contact center: B). The weight ω _n for the sentence pattern model in which these language resources are mixed and the weight μ _n for the vocabulary model are set as shown in the following equation. As an acoustic model, 2000 state 16 mixed triphone HMM was prepared, and VoiceRex was used as a speech recognition decoder. JTAG was used for morphological analysis of each language resource.

Evaluation is performed when a language model constructed based on linear interpolation from speech transcription and external language resources is used (conventional method), or when a language model created by the language model creation device 100 described above is used (proposed method 1). When the language model created by the language model creation device 200 described above was used (proposed method 2), the three cases were compared. The results are shown in Table 1.

  As shown in Table 1, the language model constructed with only the generated text (Proposed method 1) is similar to the conventional method (76.12%) (75.89%) regardless of the text automatically generated based on the probability model. ) Character correct answer accuracy was obtained. Furthermore, the language model (proposed method 2) constructed by combining the generated text and other texts by linear interpolation can achieve higher performance than the conventional method.

That is, it can be seen that the language model of the present invention provides an effective improvement effect even if the amount of voice transcription data is as small as one hour. Thus, the language model creation method of the present invention is useful as a method for constructing a language model with high performance even from a small amount of speech transcription. The language model creation method of the present invention uses a latent language model to separate a language resource into two components, a sentence-type aspect and a lexical aspect, respectively. Since the language model task is applied and constructed by appropriately mixing the language resources, it is possible to provide a language model with high performance adapted to the task.
In the above-described embodiment, the method for determining the model by comparing the uniform random number and the probability value has been described. However, the above-described method is an example, and another method using the uniform random number is used. Also good.

  When the processing means in the above apparatus is realized by a computer, the processing contents of the functions that each apparatus should have are described by a program. Then, by executing this program on the computer, the processing means in each apparatus is realized on the computer.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Further, the program may be distributed by storing the program in a recording device of a server computer and transferring the program from the server computer to another computer via a network.

  Each means may be configured by executing a predetermined program on a computer, or at least a part of these processing contents may be realized by hardware.

Claims (8)

Using a plurality of learning texts as input, a plurality of hidden sequences, which are hidden word sequences corresponding to the word strings of the respective learning texts, are generated, and a sentence model and a latent word-latent word probability are generated from the hidden sequences. An RWLM learning unit that learns a latent word language model composed of two probability distributions of vocabulary models that are word-observed word probabilities for each of the plurality of learning texts;
Using the plurality of latent language models as input, the latent language model is separated into a sentence model and a vocabulary model, and a combination of the plurality of sentence models and the vocabulary model is weighted and mixed with each other. An RWLM task adaptation unit to be generated as a model;
Using the task adaptive latent language model as an input, generating a latent word sequence, and generating a pseudo learning text from the latent word sequence,
An N-gram language model creation unit that takes the pseudo-learning text as an input and counts the frequency of all N words in the pseudo-learning text to create an LVLM-like N-gram language model;
A language model creation device comprising: The language model creation device according to claim 1,
The LMLM task adaptation unit
A model dividing means for dividing the latent language model into a sentence model and a vocabulary model;
A sentence model adaptation means for generating a task adaptive sentence pattern model in which a plurality of the sentence pattern models divided by the model dividing means are weighted and mixed;
A vocabulary model adaptation means for generating a task adaptive vocabulary model in which a plurality of the vocabulary models divided by the model division means are weighted and mixed;
A language model creation device comprising: As input learning text pseudo- created in the language model creating apparatus according to claim 1, said pseudo LWLM language model N-gram language model and counting the frequency of a word of all N tuples in the training text An LVLM-like language model creation unit created as
A language model creation unit that takes a plurality of learning texts as input and counts the frequency of all N words in each learning text to create a plurality of N-gram language models;
A mixed N-gram language model creation unit that creates the single mixed N-gram language model obtained by mixing the respective language models, using the LMLM-like language model and the plurality of N-gram language models as inputs,
A language model creation device comprising: Using a plurality of learning texts as input, a plurality of hidden sequences, which are hidden word sequences corresponding to the word strings of the respective learning texts, are generated, and a sentence model and a latent word-latent word probability are generated from the hidden sequences. An LVLM learning process of learning a latent word language model composed of two probability distributions of a vocabulary model that is a word-observed word probability for each of the plurality of learning texts;
Using the plurality of latent language models as input, the latent language model is separated into a sentence model and a vocabulary model, and a combination of the plurality of sentence models and the vocabulary model is weighted and mixed with each other. LMLM task adaptation process generated as a model,
Using the task adaptive latent language model as an input, generating a latent word sequence and generating a pseudo learning text from the latent word sequence,
An N-gram language model creation process for creating an RWLM-like N-gram language model by counting the frequencies of all N words in the pseudo-learning text using the pseudo-learning text as an input;
Language model creation method including In the language model creation method according to claim 4,
The LMLM task adaptation process is as follows:
A model dividing step of dividing the latent language model into a sentence model and a vocabulary model;
A sentence pattern adaptation step for generating a task adaptation sentence pattern model in which the plurality of sentence pattern models divided in the model division step are weighted and mixed;
A vocabulary model adaptation step for generating a task adaptive vocabulary model in which a plurality of the vocabulary models divided in the model division step are weighted and mixed;
A language model creation method comprising: As input pseudo learning text created in a language model generating method according to claim 4, the pseudo LWLM language model N-gram language model and counting the frequency of a word of all N tuples in the training text LMLM-like language model creation process created as
A language model creation process of creating a plurality of N-gram language models by counting the frequency of all N sets of words in each learning text, using a plurality of learning texts as input,
A mixed N-gram language model creation process for creating a single mixed N-gram language model obtained by mixing the respective language models with the LMLM-like language model and the plurality of N-gram language models as inputs,
Language model creation method including   A program for causing a computer to function as the language model creation device according to any one of claims 1 to 3.   A computer-readable recording medium on which any one of the programs according to claim 7 is recorded.

Images