JPWO2012165529A1 - Language model construction support apparatus, method and program - Google Patents

Abstract

The present invention extracts linguistic feature quantities from learning data, stores linguistic feature quantity origin information in which the extracted linguistic feature quantities are associated with the learning data that is the origin, and uses each linguistic feature quantity in speech recognition processing. The language feature usage statistics information indicating the degree of learning is acquired and stored, and the contribution degree of each learning data in the speech recognition processing is calculated based on the language feature statistics information and the language feature origin information, Outputs the contribution of learning data.

Description

  The present invention relates to a language model construction support apparatus, method, and program.

  In speech recognition technology that acquires text information from speech data, a component called a language model is used. This component is used by the speech recognizer to select a more linguistically valid one from a plurality of recognition result text hypotheses. The language model widely used today is an N-gram language model. The N-gram language model is constructed by statistically learning the likelihood that N word chains appear from a large amount of text data. Since the N-gram language model operates based on statistical information obtained from learning data, in a state where learning data belonging to a specific domain is at hand, speech using a language expression belonging to that domain has high accuracy. However, the recognition accuracy for speech using language expressions that do not belong to the domain is low.

For example, Patent Document 1 divides text data into a plurality of clusters, generates a language model for each cluster, evaluates each language model using evaluation data, and selects a language model selected based on the evaluation result. An apparatus for creating a language model suitable for a target application without requiring text data appropriate for the target application by integration or the like is disclosed.

Japanese Patent No. 4537970

  However, in the device of Patent Document 1, the recognition accuracy for the target domain of the finally obtained language model is higher than that of other cluster language models, but there is no guarantee that good recognition accuracy can be achieved. What is guaranteed by this method is that the recognition accuracy for the target domain of the finally obtained language model is higher than that of other cluster-specific language models.

  In order to obtain better recognition accuracy, it is desirable to use learning data that has a high effect of adapting the language model to the target domain, and it is desirable to realize a mechanism for allowing the user to register such learning data. However, it has been difficult for the user to understand what learning data is effective in improving the accuracy of speech recognition.

  The present invention has been made in view of the above problems, and provides a language model construction support apparatus, method, and program for supporting registration of highly effective learning data for adapting a language model used in speech recognition processing to a target domain. For the purpose.

  Another object of the present invention is to provide a language model construction support apparatus, method, and program that allows a user to easily grasp what learning data is effective in improving the accuracy of speech recognition. And

  The present invention includes a language feature amount extraction unit that extracts a language feature amount from learning data, a unit that stores language feature amount origin information in which the extracted language feature amount is associated with the learning data that is the origin, speech recognition In the speech recognition processing, based on the language feature quantity statistical information and the language feature quantity origin information, means for acquiring and storing language feature quantity use statistical information indicating the degree of use of each language feature quantity in the processing A language model construction support apparatus comprising: a contribution calculation unit that calculates a contribution of each learning data; and a unit that outputs the calculated contribution of the learning data.

  The present invention is a language model construction support method that extracts a language feature amount from learning data, stores language feature amount origin information in which the extracted language feature amount is associated with learning data that is the origin, and stores speech information Language feature usage statistics information indicating the degree to which each language feature is used in the recognition process is acquired and stored, and based on the language feature statistics information and the language feature origin information, The language model construction support method is characterized in that the contribution of learning data is calculated and the calculated contribution of learning data is output.

  The present invention relates to a process for extracting a language feature quantity from learning data, a process for storing language feature quantity origin information in which the extracted language feature quantity is associated with the learning data that is the origin, and a speech recognition process. Processing for acquiring and storing linguistic feature quantity usage statistical information indicating the degree to which the linguistic feature quantity is used, and based on the linguistic feature quantity statistical information and the linguistic feature quantity origin information, It is a program for executing a process for calculating a contribution degree and a process for outputting the calculated contribution degree of learning data.

  ADVANTAGE OF THE INVENTION According to this invention, registration of the learning data with the high effect of adapting the language model used by the speech recognition process to a target domain can be supported.

FIG. 1 is a block diagram of a language model construction support apparatus according to the first embodiment of the present invention. FIG. 2 is a flowchart for explaining the counting phase. FIG. 3 is a flowchart for explaining the learning phase. FIG. 4 is a flowchart for explaining the recognition phase. FIG. 5 is a flowchart for explaining the analysis phase. FIG. 6 is a diagram illustrating another configuration example of the language model construction support apparatus according to the first embodiment. FIG. 7 is a block diagram of a language model construction support apparatus according to the second embodiment. FIG. 8 is a diagram illustrating data stored in the language feature amount origin DB. FIG. 9 is a block diagram of a language model construction support apparatus according to the third embodiment. FIG. 10 is a block diagram of a language model construction support apparatus according to the fourth embodiment. FIG. 11 is a block diagram of a language model construction support apparatus according to the fifth embodiment. FIG. 12 is a block diagram of a language model construction support apparatus according to the seventh embodiment.

  Embodiments of the present invention will be described below with reference to the drawings.

(First embodiment)
FIG. 1 is a block diagram of a language model construction support apparatus according to the first embodiment of the present invention. As illustrated, the language model construction support apparatus according to the present embodiment includes a language feature quantity extraction unit 101, a language feature quantity source storage unit 102, a language model learning unit 103, a speech recognition unit 104, and learning data. A contribution analysis unit 105.

  When the language feature quantity extraction unit 101 receives learning data of a language model as an input, the language feature quantity extraction unit 101 extracts (calculates) a language feature quantity from the learning data according to a given procedure.

  The language feature quantity origin storage unit 102 stores the language feature quantity extracted by the language feature quantity extraction unit 101 in a database (language feature quantity origin DB) in such a way that the learning data that is the origin can be tracked. For example, information (language feature amount origin information) that “the word“ shiba inu ”appears 20 times in the learning data d1 and 15 times in the learning data d2” is recorded.

  The language model learning unit 103 uses the language feature amount extracted by the language feature amount extraction unit 101 to construct a language model based on a given procedure. For example, in the case of a general word trigram (3-gram) language model, the number of appearances for each group of three words that appear successively is the language feature. Divide this number of occurrences by the number of first two words (context) that appear in succession. A probabilistic language model can be constructed by calculating this for all given language features.

  The speech recognition unit 104 uses the language model constructed by the language model learning unit 103 and performs speech recognition processing on a separately input speech signal according to a given procedure. In addition, the voice recognition unit 104 records each language feature amount referred to in the voice recognition process. Thereby, language feature amount usage statistical information indicating a reference count for each language feature amount in the speech recognition processing is obtained.

  The learning data contribution analysis unit 105 extracts a language feature amount according to a given algorithm, based on the language feature amount use statistical information recorded by the speech recognition unit 104 and appropriately referring to the language feature amount source storage unit 102. The degree of contribution to the speech recognition process is calculated for each learning data input to the unit 101.

  Next, the operation of the language model construction support apparatus according to this embodiment will be described. The processing in the language model construction support apparatus is mainly composed of four independent phases. That is, a total phase, a learning phase, a recognition phase, and an analysis phase. There is a direct dependency between each phase of aggregation → learning, learning → recognition, recognition → analysis. Each phase operates using the latest processing result of the phase having a direct dependency relationship. Typically, each phase is executed in the order of tabulation → learning → recognition → analysis, and hence the description will proceed in this order. However, this is only an example, and it is not always necessary to activate the next phase upon completion of a certain phase. For example, these phases may be executed at arbitrary timings.

  For example, the learning phase may operate in response to the completion of the aggregation phase, may be automatically activated immediately before the recognition phase is activated, or may be explicitly activated by the user, You may make it start with a timer for every predetermined time (every minute to every hour, every night etc.). The recognition phase is executed in response to a user's voice recognition processing request. The analysis phase may operate in response to the completion of the recognition phase, may be explicitly activated by the user, or may be activated by a timer at predetermined time intervals.

  The aggregation phase will be described. The counting phase is executed each time new learning data is given from the user, and the language feature source database is sequentially updated. Hereinafter, the details of the aggregation phase will be described with reference to FIG.

  First, learning data is input by the user (A1). The learning data is a lump of text data, typically a single document file. If the text data can be extracted, the specific format of this file is not limited. Further, text data directly input from a text input form of a web page may be handled as one learning data.

  One learning data is preferably composed only of language expressions belonging to a single domain, but may include language expressions belonging to a plurality of domains. However, learning data that uniformly includes language expressions belonging to different domains is compared with learning data that includes only language expressions belonging to a specific domain, and voice recognition processing for speech utterances made in accordance with the expression format belonging to that specific domain. There is a case where the contribution to is low.

  For example, when the voice is a daily business report, it is desirable that the learning data given is the daily business report itself (the “business daily report” domain), but this is the product brochure (“product introduction domain”). If given as one piece of learning data, the linguistic expression specific to the “business daily report” domain hardly increases despite the increase in the absolute amount of language expressions included in the learning data. The contribution to is reduced.

  The learning data preferably includes many linguistic expressions that are actually used for speech utterance, but such linguistic expressions may not be included. However, such learning data may reduce speech recognition accuracy.

  The language feature quantity calculation unit 101 extracts a language feature quantity from the given learning data (step A2). For example, after performing morpheme analysis processing and dividing into words (morpheme) units, the number of appearances is counted in a chain unit of N words. As other language feature quantities, the appearance frequency of each word (unigram frequency) and the appearance frequency in units of meta information (part of speech, semantic class, etc.) to which each word or word chain belongs can be considered. Also, a phrase composed of a combination of a plurality of words, a subtree of a syntax tree based on dependency analysis, or the like may be used. What kind of feature value is to be extracted is determined in advance according to the finally generated language model and the voice recognition unit 104 described later.

  The language feature quantity source storage unit 102 stores the appearance frequency of the language feature quantity extracted in step A2 in a form in which the extracted learning data can be traced (step A3). For example, a serial number of learning data registered in the language feature quantity origin storage unit 102 is adopted as an ID number representing learning data, and a list having an ID number and an appearance frequency pair as an element for each language feature quantity. Record in the language feature source database. This completes the counting phase.

  After the aggregation phase is performed at least once, the language model learning phase is executed. (1) After the completion of the totaling phase, (2) every predetermined time (every few minutes to every several hours) and every regular period, (3) the user is explicitly instructed to start the learning phase If, etc., is considered. In the case of a configuration other than interlocking with the aggregation phase, a database that temporarily accumulates output results in the aggregation phase may be used, or a language feature amount origin database may be diverted. In any case, after the previous learning phase, if no change is recognized in the learning data or the language feature extracted from the learning phase, the learning phase may be terminated without any processing.

  Hereinafter, the details of the learning phase will be described with reference to FIG.

  The language model learning unit 103 collects aggregated language feature amounts based on the given setting (step A4). Specifically, for example, language feature amounts extracted from all known learning data are added. At this time, the frequency of language features that can be regarded as the same is added beyond the learning data, but for example, when learning an N-gram language model, a weighted sum with a weight value separately provided for each learning data is used. You may take Further, this weight may be automatically calculated using some algorithm. Since these are methods generally used for task adaptation of the language model, details thereof are omitted.

Next, the language model learning unit 103 learns a language model using the collected language feature amount (step A5). For example, in the case of an N-gram language model, the occurrence frequency of N chains of words collected as language feature amounts is expressed as the occurrence frequency of (N-1) chains by the first (N-1) words constituting the N chain. Get by dividing by. That is, when the frequency of occurrence of a word chain _{w i-N + 1 ... w} i-1 w i and _{^{N (w i i-N +}} 1), the words chain _{w i-N + 1 ... w} i-1 is obtained next to w _The conditional probability P (w _i | w ⁱ _{i−N + 1} ) in which _i appears is obtained by (Equation 1).

  This completes the learning phase and builds the language model. In general, a process (smoothing) for assigning some probability value to a word chain having a low appearance frequency is also performed.

  After the learning phase is completed at least once, the recognition phase is executed. The recognition phase is executed every time a voice recognition process is requested by the user. Hereinafter, the operation in the recognition phase will be described with reference to FIG.

  First, voice signal data is input to the voice recognition unit 104 by the user. The voice recognition unit 104 analyzes the voice signal data based on a predetermined algorithm, and outputs a character string, a word string, or the like determined to be most appropriate (step A6). In this speech recognition process, a language model constructed by the learning phase is used.

  Although there are various methods for the specific processing procedure of the speech recognition processing in the speech recognition unit 104, it is generally performed as follows.

  First, an acoustic feature amount is calculated from the input voice signal. The acoustic feature amount is a vector of acoustic feature amounts such as cepstrum, power, and Δ power that are used as clues when a person perceives speech. By comparing the acoustic feature quantity sequence obtained in this way with the acoustic feature quantity pattern distribution of each phoneme recorded in a component called an acoustic model, possible phoneme sequence combinations are generated. The closeness of the cumulative distance between the pattern distribution and the phoneme sequence is called acoustic likelihood.

  Next, a possible word sequence (hypothesis) is obtained by comparing each phoneme sequence with a dictionary. Then, the validity of each word sequence is evaluated by the language model constructed by the language model learning unit 103, and a numerical evaluation result (language likelihood) is obtained. In the case of a probabilistic language model such as an N-gram language model, the language likelihood is the appearance probability of a word sequence. Finally, a value obtained by mixing the acoustic likelihood and the language likelihood with a given weight is calculated for each hypothesis word sequence, and the hypothesis having the highest value is output as a recognition result. In fact, since all hypotheses cannot be searched, hypotheses with low likelihood are discarded according to the input of the speech signal. Further, the search process for the maximum likelihood hypothesis is generally solved as a minimum cost route search problem by using the inverse of the likelihood. In the present invention, a specific algorithm for speech recognition is not particularly limited, but at least a language model used therein must be constructed by the learning phase described above.

  In addition, the voice recognition unit 104 records each language feature amount referred to in the voice recognition process. That is, the speech recognition unit 104 calculates language feature usage statistics information after completion of the speech recognition processing or in accordance with the progress of the speech recognition processing, and inputs this to the learning contribution analysis unit 105 (step A7). .

  For example, words and word chains included in the finally obtained maximum likelihood word sequence are recorded and output separately from the maximum likelihood word sequence. Not only the maximum likelihood word sequence but also the appearance frequency of words and word chains in N hypotheses (N-best) with the highest likelihood may be targeted. Alternatively, the frequency of occurrence of words or word chains included in the lattice structure (word graph) of the hypothesized word sequence group held inside the speech recognition unit 104 when the maximum likelihood word sequence is obtained Good. In addition to the final higher-level hypothesis, all words or word chains that refer to the language model during the search and the number thereof may be recorded and output.

  Moreover, when recording them, you may record and output together the acoustic likelihood of the phoneme sequence which comprises the word or word chain. Predetermined semantic analysis processing is performed on the most likely word sequence or N-best sequence or word graph, and semantic units (phrases or specific expressions) consisting of one or more word chains are extracted and the number of occurrences is counted. -You may record and output. Or, meta information (part of speech, semantic class) possessed by a word or word chain may be aggregated, recorded, and output as a unit. Further, the dependency structure or co-occurrence information of words or word strings may be aggregated, recorded, and output.

  Whichever statistical information is used, the linguistic feature quantity to be counted must be determined by referring to the linguistic feature quantity origin storage unit 102 and how much it has appeared in which learning data in the past. Don't be. Typically, this means that the language feature quantity extracted by the language feature quantity extraction unit 101 and recorded by the language feature quantity source storage unit 102 is targeted. However, a secondary feature amount that is obtained by calculation from a plurality of data recorded by the language feature amount source storage unit 102 may be used. Specifically, even if only the word trigram is recorded in the language feature quantity source storage unit 102, it is possible to derive the language feature quantity related to the word bigram or the part-of-speech trigram by appropriately integrating the word trigram. Therefore, such statistical information on language features may be used. In addition, a plurality of different algorithms may be used for the language feature quantity extraction unit 101 as long as consistency is maintained. This completes the recognition phase.

  After at least one recognition phase is completed, the analysis phase is executed. The analysis phase activation timing includes (1) after completion of the recognition phase, (2) every predetermined time (every few minutes to every few hours), (3) when the user explicitly instructs activation, etc. Conceivable. When the configuration other than the activation is performed in conjunction with the recognition phase, a database that temporarily stores the usage statistical information of the language feature amount that is the output result in the recognition phase may be used, or the learning data contribution analysis unit 105 You may enter directly into. In any case, if no change is recognized in the language feature usage statistics after the previous recognition phase, the analysis phase may be terminated without any processing. Hereinafter, the operation in the analysis phase will be described with reference to FIG.

  The learning data contribution degree 105 uses statistical information for each language feature amount output from the speech recognition unit 104, and refers to the language feature amount origin unit 102, and how much each learning data is referred to in the speech recognition processing. The contribution is calculated using a given algorithm.

  An example of a contribution calculation algorithm is given. The voice recognition unit 104 acquires the number of appearances of each word included in the maximum likelihood word sequence in the voice recognition process. For example, if the recognition result word string for the utterance “Today is sunny and tomorrow is rain” is “Today / Has / That / Tomorrow / Has / Rain / In”, “Today” “That” “Tomorrow” “Rain” The statistical information that “de” appears once and “ha” appears twice is input to the learning data contribution analysis unit 105. The learning data contribution analysis unit 105 stores statistical information input in the past, and adds the obtained statistical information. As a result, for example, “Today” is 30 times, “That” is 100 times, “Tomorrow” is 40 times, “Rain” is 10 times, and “Ha” is 500 times. Hereinafter, a set of word appearance information at a certain time t is expressed as Et, and the cumulative number of appearances of the word w at Et is expressed as Et (w).

  The learning data contribution analysis unit 105 refers to the language feature amount origin storage unit 102 and calculates how many words appear in each of the learning data. For example, the word “today” is searched for information such as 200 times in the learning data d1, 20 times in the learning data d2, and none in the learning data d3. Hereinafter, the number of occurrences of the word w in a certain learning data c is expressed as F (w | c). Then, the contribution Contribute (t | c) of the learning data c at a certain time t is expressed as (Equation 2).

  (Expression 2) indicates that the learning data that gives more words included in the recognition results obtained in the past has a higher contribution to the past speech recognition. Here, α is a weight for each learning data, and is basically adjusted to the value used in step A4. When learning is performed without adding any particular weight, α = 1 for all learning data.

  In this case, the number of times used in the recognition process is not considered, but if this is taken into consideration, (Equation 3) is obtained.

  In addition, the above formula simply treats all words fairly as calculation targets, but for example, the case particles “ga”, “ha”, etc. are excluded from the calculation because they work in the same way regardless of the learning data. Or may be adjusted so as to reduce the influence by multiplying by an appropriate coefficient. If the weight for each word is β (w), for example, (Equation 4) is obtained.

  There are two methods for assigning weights for each word: a method in which weight values are given in advance for each word, and a method for obtaining values during speech recognition processing. The former is determined in advance, for example, giving a high weight to proper nouns and giving a low weight to particles and common nouns. The latter is obtained from, for example, the likelihood of each word of the recognition result and the magnitude of the score. For example, so-called reliability such as word posterior probability can be used as a weight as it is.

  Further, a method of considering the size of learning data is also conceivable. For example, if a word referred to during the recognition process appears 10 times for learning data consisting of 5000 words and 10 times for a document consisting of 100 words, the latter is learning data closer to the content of the speech. Probability is high. This is expressed by, for example, (Formula 5).

  Here, y indicates a word included in the learning data c. Since w is a word referred to during the recognition process, attention must be paid to the fact that the population is different. As described above, this expression indicates that the contribution is obtained based on the appearance probability instead of the number of appearances of each word in the learning data. Since the learning data finally affects the appearance probability of each language feature in the language model, it can be said that this is a method that reflects this.

  There is another way of thinking about the size of the learning data. For example, in the above-mentioned learning data A, even if the appearance probability of a certain word w is surely small, w appears only in the learning data A when viewed as the whole language feature amount obtained from all the learning data. If so, the contribution is considered significant. For this, for example, a method in which the final contribution is obtained by linearly summing Equations 4 and 5 using appropriate weights can be considered.

  Also, since the appearance frequency distribution of words tends to appear extremely large and small, a logarithmic value may be used instead for each appearance number.

  Even when a language feature other than words is used, the degree of contribution can be calculated with the same algorithm. For example, when three word chains (3-gram) are used, the speech recognition means 104 is “<S> / Today / ha /”, “Today / ha / that”, “ha / that” in the above example. The number of appearances (one each) for the seven word chains “/ Tomorrow”, “Tomorrow / Ha / Rain”, “Ha / Rain / De”, and “Rain / De / <E>” is tabulated. Here, <S> and <E> are special symbols representing the beginning and end of the sentence, respectively. “De / <E> / <?>” May be counted as the eighth word chain, but in this case, any word entry that does not exist in the third word (= cannot be subtracted from the phoneme string) <? > Is determined in advance and assigned. The subsequent contribution calculation algorithm is almost the same as the case where a word is a language feature, and is omitted.

  If the contribution for each learning data is calculated, the system outputs it in a predetermined format (A9). Specifically, for each learning data, a list of contributions calculated using linguistic feature quantity usage statistics obtained from all speech recognition processes performed in the past is encoded and output in a predetermined data format. .

  Moreover, you may output the contribution calculated using only the language feature-value utilization statistics obtained from the speech recognition process of the specific user. You may output the contribution with respect to the speech recognition process of the group which consists of one or more specific users. Or you may calculate the contribution with respect to the speech recognition process performed within the past certain time. For example, it is possible to display a graph (such as a line graph) by sequentially outputting the contribution for each learning data on a daily basis. Alternatively, the learning data may be collectively output for each registered user instead of each learning data. For example, if a user registers four pieces of learning data, and the contributions to all speech recognition processes in the most recent week are 10%, 15%, 2%, and 5%, the learning data registered by the user The degree of contribution by can be calculated as follows: 10 + 15 + 2 + 5 = 32%. This completes the analysis phase.

  As described above, according to the present invention, there is provided means for recording usage statistics of language feature values referred to during speech recognition processing, and how each language feature value is determined for each given learning data. Since it has a configuration in which it can be referred later, it is the learning data that contains more language features that are actually referred to more in the speech recognition process, and what is its relative size Can be calculated and presented to the user.

The system configuration shown in FIG. 1 is an example. For example, as shown in FIG. 6, two language feature extraction units 101 are arranged, one in the language feature quantity source storage unit 102, and the other in language model learning. It is good also as a structure connected to the part 103, respectively. For example, the language model is an N-gram model, but when using dependency as a language feature, there is almost no information shared between language feature collection and language model construction. The configuration of FIG. 6 may be applied.

(Second Embodiment)
Next, a language model construction support apparatus according to the second embodiment of the present invention will be described. FIG. 7 is a block diagram of a language model construction support apparatus according to the second embodiment. This language model construction support apparatus has substantially the same configuration as that of the first embodiment, except that a language model before update can be given to the language model construction unit. Hereinafter, a description will be given focusing on differences from the first embodiment.

  Step A5 in the learning phase is a step of actually constructing a language model. In the second embodiment, a new language model (updated language model) is constructed by simultaneously using a language model (pre-update language model) given separately in addition to the language feature values tabulated in step A4.

  The language model learning unit 203 registers a set of language feature values used for constructing the pre-update language model in the language feature value source storage unit 202. In this case, since the origin of these language feature amounts is the pre-update language model itself, it can be appropriately distinguished from other learning data when output in step A9.

  There are, for example, two methods for extracting the language feature used for the construction from the pre-update language model.

  In the first method, when learning the pre-update language model, it is related to the language model (which can be regarded as a table returning the language likelihood using the language feature amount as a key) and This is a method of storing a set together. For example, if the pre-update language model itself is constructed by a system according to the first embodiment of the present invention, it is easy to store such information.

  The second method is a method of restoring the number of appearances of each language feature amount from a language likelihood for each language feature amount included in the pre-update language model using a given algorithm. Any method can be used for the frequency restoration algorithm. For example, Reference 1 “Construction of language model fusion tool using complementary back-off, IPSJ Research Report. SLP, Spoken Language Information Processing 2001, 11, p49” -54 "introduces a restoration method using the fact that the N-gram language model is a product of conditional probabilities. Although this is an approximate value, there is no particular problem in accuracy in use in the present invention.

  In addition, various methods are conceivable as processing methods for constructing a language model by using the pre-update language model in combination with the language feature values totaled in step A4. Here, two typical methods will be described. Which of these methods is used can be arbitrarily set.

  The first language model update method is to take the sum (or weighted sum) of the set of language feature values used to construct the pre-update language model and the set of language feature values tabulated in step A4. is there. In the above-mentioned document 1, this is called corpus coupling. In the present embodiment, as described above, since the process of extracting the number of appearances of language feature values from the pre-update language model is performed, this method is particularly suitable.

  The second language model update method is a linear sum of language likelihoods between a language model constructed by the same method as in step A5 in the first embodiment and the language model before update from the aggregated language feature quantities. Is something to take. Although there are various methods for taking a specific linear sum, in literature 1, for a language feature that does not appear in one language model, a method of giving zero as a probability value and a probability estimated from the other language model The method of giving the value is described. This method has a higher processing speed than the first method.

  According to the second embodiment of the present invention, a language model is constructed from scratch because it is constructed to further add language feature values obtained from learning data added later to an existing language model. It is possible to obtain a larger and more robust language model than this. Even when such a language model is used, it is possible to correctly determine the contribution of learning data added later.

  Next, a system using task daily report input as a task will be described as an example of the present embodiment. In addition to past (handwritten) business daily reports, learning data includes product brochures, emails, etc., and one file is used as one learning data. The language model to be generated is a word trigram model, and the language feature quantity extraction unit 201 counts the appearance frequency of each of the three word chains. An example of data stored in the language feature quantity origin DB by the language feature quantity origin storage unit 202 is shown in FIG.

  With such data, the appearance frequency for each learning data can be acquired using the trigram as a key. It is assumed that the pre-update language model is attached with a trigram appearance frequency table that was input when the model was learned. The language model learning unit 203 merges the appearance frequency table of the pre-update language model (a table in which the appearance frequency can be subtracted using the trigram as a key) with the table stored in the language feature amount source storage unit 202 (increase by one column). Thereafter, the trigrams are referred to one by one in the table of the language feature amount origin storage unit 202, and the appearance frequencies of all lines are summed and stored. Subsequent probability calculation processing operations are the same as in the first embodiment, and the calculated trigram → probability value table is output as an updated language model.

  Similar to the first embodiment, the speech recognition unit 204 calculates a maximum likelihood word string and outputs it to the display device. On the other hand, each word included in the maximum likelihood word string is input to the learning data contribution analysis unit 205. The learning data contribution analysis unit 205 counts the cumulative number of words received (cumulative usage frequency). As the retention method, a table that can use the trigram and the execution time of the speech recognition processing as a key and can use the table at the execution time is used, as in the language feature amount source storage unit 202. Thereby, for example, a processing operation of deleting data after one month from the execution time may be performed. Further, a user account indicating the user who executed the voice recognition process may also be recorded.

  When the user presses the “view learning data usage” button from the UI (user interface) screen, the learning data contribution analysis unit 205 calculates the contribution based on Equation 4 from the count number held, A list is generated for each learning corpus and output to the display device. Here, both α and β are always 1. In addition, when the user inputs the number of days or months in the “use period to be calculated” field from the UI screen, the learning data contribution analysis unit 205 executes the entry when obtaining the cumulative use frequency of each trigram. Ignore entries whose time does not belong to the specified period. When the user selects a specific user from the “user to be calculated” list box from the UI screen, the entry that is not registered together with the user account indicating the user is similarly ignored. A plurality of users can be registered using a selectable list box or can be selected for each user group separately defined, so that a plurality of users belonging to the selected user user group can be selected. Similarly, entries registered with a user account that is not one of the selected users are ignored.

  When registering learning data, a plurality of attributes may be assigned to each learning data. For example, a user count of registered users of learning data, a registration date and time, or a tag (arbitrary label) can be considered. The user can select learning data having a specific attribute value as a contribution calculation target. For example, a control “view the usage status of learning data whose registrant is XX” (XX is a list box or text field) is arranged on the UI, and is selected from here. Further, the contribution may be calculated and displayed as a learning data cluster for each attribute. For example, when a button “view learning data usage status at every registration time” is arranged on the UI screen and the user selects it, the learning data is used for a specific period, for example, 1 After classifying in units of months and making each a learning data cluster, the contribution is calculated in units of learning data clusters.

(Third embodiment)
FIG. 9 is a block diagram of a language model construction support apparatus according to the third embodiment. This language model construction support apparatus has the same configuration as that of the second embodiment, but also performs speech recognition in the pre-update language model, takes a difference in language feature from the post-update model, And a means for calculating a contribution degree from the above.

  In the second embodiment, when the contribution level of the pre-update language model is large, the contribution degree to the individual learning data is relatively small and the range of superiority and inferiority is also small. This processing method is particularly desirable when paying attention to the difference in contribution between learning data. As a case where the difference in the contribution level of learning data is particularly important, for example, a reward such as a reward is given to a user who has added learning data with a high contribution degree. In the second embodiment, a similar value can be obtained by assuming that the number of appearances of each language feature in the pre-update language model is zero in Step A8, but this is more strictly the language before and after the update. The influence of learning data group on the model can be evaluated.

(Fourth embodiment)
FIG. 10 is a block diagram of a language model construction support apparatus according to the fourth embodiment. This embodiment corresponds to a case where a correct transcription text (correct text) for a word spoken by a speaker is given. The language feature quantity extraction unit 501, language feature quantity source storage unit 502, and learning data contribution degree analysis unit 503 included in the model construction support apparatus of the present embodiment have substantially the same functions and functions as the corresponding components in the first embodiment. Although having a configuration, the learning data contribution degree analysis unit 503 extracts a language feature amount from the input correct text, and calculates a contribution degree using the statistical information of the extracted language feature amount as usage statistical information.

(Fifth embodiment)
FIG. 11 is a block diagram of a language model construction support apparatus according to the fifth embodiment. This embodiment has substantially the same configuration as that of the first embodiment, but the learning data contribution analysis unit 105 is input for language feature usage statistics recorded by the speech recognition unit 104 and for each speech. The difference is that the degree of contribution of learning data is calculated based on correct text and the like.

  For example, a conference room recording voice transcription assisting system uses a workflow in which speech recognition is first performed, errors are corrected from the recognition result text, and the final transcription is finished. In such a case, the recognition result and the correct text can be compared. Therefore, for example, the learning data contribution analysis unit 105 multiplies the usage frequency by +1 for the language feature amount common to the speech recognition result and the correct text, and -1 for the unmatched language feature, The degree of contribution is obtained by a method such as 5. Further, the correct text itself is not given, and the recognition result text may be “almost correct answer”, “many mistakes”, or “almost wrong”, or the degree of correct answer may be given. In this case, the level of the correct answer (three levels in the above example) is used as a weight, for example, a numerical value such as + 1,0, -1 is multiplied by the language feature usage statistical information. For example, if the word “NEC” appears 5 times for the “almost correct” utterance and 3 times for the “almost wrong” utterance, the contribution may be calculated by subtracting it as 2 times. Good. Furthermore, error correction information may be given only for a part of the recognition result. As an example of this, there is a case where a word that is a recognition error is designated and a correct answer is selected from the presented lower candidates, such as Kana-Kanji conversion. In this case, it can be determined that the corrected word is clearly an error, but it cannot be determined whether the other words were correct or whether it was considered an error but acceptable. Therefore, the positive effect brought about by the uncorrected word can be made mild by regarding the corrected word as -2 and the uncorrected word as +1. Of course, in the case where information is explicitly input that a certain word is correct, +2 may be given to show a higher effect than a word that is not explicitly shown as an error or correct answer. .

(Sixth embodiment)
In each of the first, second, third, and fifth embodiments described above, the hypothesis was frequently adopted as a hypothesis during the speech recognition process, but the recognition result (and / or the top N candidates) did not remain in the end. A language feature amount may be searched and a penalty may be applied to the contribution degree of learning data including many such language feature amounts. For example, if the final recognition result is “Prime Prime Minister” and the speech recognition unit frequently refers to “cancer” as a hypothesis on the way, the learning data containing many words “cancer” It is assumed that it may belong to a domain (for example, “medical” domain) that is not related to (in this example, “political” domain). Since learning data belonging to another domain may have a negative effect on the utterance of the target domain, the contribution of such learning data is slightly deducted and presented to the user. For example, a word that has not been left in any of the top K hypotheses with high likelihood in spite of being adopted N times as a hypothesis, has − (γ / N) in the language feature usage statistical information. Multiply as a penalty. γ is a given constant and adjusts the strength of the penalty effect.

(Seventh embodiment)
FIG. 12 is a block diagram of a language model construction support apparatus according to the seventh embodiment. The language model construction support apparatus of this embodiment includes a plurality of different speech recognition units. Although learning data is given collectively, there are a plurality of speech recognition units 104, and language models are learned (updated) separately. All of the voice recognition processes may be moved simultaneously, or may be moved by any one (or a plurality) explicitly designated by the user. With such a system configuration, different language feature usage histories (language feature usage statistics) are obtained for each speech recognition unit, so that contributions can be obtained independently of each other.

  Thereby, it can be shown how much the learning data contributed to which speech recognition unit. For example, by assigning a voice recognition unit (and language model) for each domain or for each application (for example, for voice dictation, voice map search, etc.), the user can contribute to which domain / application the learning data You can know what to do. You may receive from a user the input which designates the linguistic feature-value statistics information used for calculation of a contribution.

  When a plurality of voice recognition units are operated simultaneously, the final recognition results are often combined into one by some method. For example, using the ROVER method disclosed in `` A post-processing system to yield reduced word error rate: Recognizer Output Voting Error Reduction (ROVER), In Proceedings IEEE Automatic Speech Recognition and Understanding Workshop, pp.347-352 '' Also good. A plurality of them may be presented to the user simultaneously.

(Eighth embodiment)
The language model construction support apparatus according to the eighth embodiment has the same configuration as that of each of the first to seventh embodiments described above, and performs analysis in units of a plurality of learning data clusters when calculating the contribution. . As an expression for calculating the contribution degree, summation of “each corpus belonging to the learning cluster” is further calculated using Expressions 2 to 5. As the learning data cluster, for example, various settings such as those registered by the same user, those of the same department, those explicitly selected by the user, those registered in a predetermined period (for example, one month) are used. be able to. The language model construction support apparatus according to the eighth embodiment has the same configuration as that of each of the first to seventh embodiments described above, except that learning data is divided into several clusters and recorded and referenced. Different. For example, the cluster may be one cluster for each user, or one registered from users belonging to the same department may be one cluster, or divided into fixed periods (for example, one month) according to the registered time. Also good. The calculation of the contribution degree is performed for each cluster in the same procedure as Expressions 2 to 5, and then the sum (weighted sum) is obtained using a given weight value. Thus, for example, if clustering is performed at regular intervals, it is possible to distinguish between learning data that has a particularly strong influence and learning data that does not have a strong influence. Of course, the way of dividing the cluster is not limited to the above.

  Although the present invention has been described with reference to the preferred embodiments, the present invention is not necessarily limited to the above-described embodiments, and various modifications can be made within the scope of the technical idea.

  The language model construction support apparatus according to the embodiment of the present invention described above may be realized by a computer including a control unit such as a CPU (Central Processing Unit), a storage unit such as a memory and a hard disk, a communication unit, and the like. Language feature extraction unit, language feature source storage unit, language model learning unit, speech recognition unit, and learning data contribution analysis unit are realized by the CPU reading and executing the operation program stored in the storage unit. It may also be configured by hardware. Only some functions of the above-described embodiments can be realized by a computer program. The computer program may be recorded on a computer-readable storage medium. The computer program may be loaded from a recording medium into a memory, or may be downloaded to a computer through a network and loaded into the memory.

  A part or all of the above-described embodiment can be described as in the following supplementary notes, but is not limited thereto.

(Appendix 1)
Language feature extraction means for extracting a language feature for each learning data;
Means for storing language feature quantity origin information in which the extracted language feature quantity is associated with learning data that is the origin of the extracted language feature quantity;
Means for acquiring and storing linguistic feature quantity usage statistical information indicating the degree of use of each linguistic feature quantity in speech recognition processing performed in the past;
A contribution degree calculating means for calculating a contribution degree of each learning data in the speech recognition processing based on the language feature quantity statistical information and the language feature quantity origin information;
Means for outputting the calculated contribution of the learning data;
A language model construction support apparatus comprising:

(Appendix 2)
The language model construction support according to appendix 1, wherein the contribution degree calculation means calculates a value corresponding to the appearance degree of the language feature amount used in the speech recognition process in each learning data as a contribution degree. apparatus.

(Appendix 3)
Based on at least one of correct text for the speech to be processed in the speech recognition process, information on the correctness of the speech recognition result of the speech, and correction information for the speech recognition result of the speech, the language feature usage statistics information Weight,
The language model construction support apparatus according to Supplementary Note 1 or 2, characterized in that:

(Appendix 4)
Updating or constructing a language model used in the speech recognition processing based on the language feature amount acquired by the language feature amount extraction unit;
4. The language model construction support apparatus according to any one of supplementary notes 1 to 3, wherein

(Appendix 5)
Further comprising speech recognition means for performing speech recognition on speech data using the language model and outputting the language feature usage statistics information;
5. The language model construction support apparatus according to any one of appendices 1 to 4, wherein

(Appendix 6)
Switching or limiting at least one of the learning data to be calculated for contribution and the language feature usage statistical information used for calculating the contribution according to the user's request,
6. The language model construction support apparatus according to any one of appendices 1 to 5, wherein

(Appendix 7)
The difference between the linguistic feature quantity statistical information acquired in the speech recognition process performed using the language model before the update and the linguistic feature quantity statistical information acquired in the speech recognition process performed using the updated language model is obtained. The degree of contribution of learning data is calculated using the difference,
The language model construction support device according to any one of appendices 1 to 6, wherein

(Appendix 8)
A language model construction support method,
Extract language features for each learning data,
Stores language feature origin information that associates the extracted language feature with the learning data that is its origin,
Acquiring and storing language feature usage statistics information indicating the degree to which each language feature was used in speech recognition processing performed in the past;
Based on the linguistic feature quantity statistical information and the linguistic feature quantity origin information, the contribution of each learning data in the speech recognition processing is calculated,
Outputting the calculated contribution degree of the learning data;
A language model construction support method characterized by this.

(Appendix 9)
9. The language model construction support according to appendix 8, wherein in the calculation of the contribution level, a value corresponding to the appearance level of the language feature amount used in the speech recognition processing in each learning data is calculated as the contribution level. Method.

(Appendix 10)
Based on at least one of correct text for the speech to be processed in the speech recognition process, information on the correctness of the speech recognition result of the speech, and correction information for the speech recognition result of the speech, the language feature usage statistics information Weight,
The language model construction support method according to appendix 8 or 9, characterized in that.

(Appendix 11)
Updating or constructing a language model used in speech recognition processing based on the acquired language feature amount,
The language model construction support method according to any one of appendices 8 to 10, characterized in that:

(Appendix 12)
Performing speech recognition on speech data using the language model, and outputting the language feature usage statistics information;
The language model construction support method according to any one of appendices 8 to 11, wherein

(Appendix 13)
Switching or limiting at least one of the learning data to be calculated for contribution and the language feature usage statistical information used for calculating the contribution according to the user's request,
13. The language model construction support method according to any one of appendices 8 to 12, characterized in that:

(Appendix 14)
The difference between the linguistic feature quantity statistical information acquired in the speech recognition process performed using the language model before the update and the linguistic feature quantity statistical information acquired in the speech recognition process performed using the updated language model is obtained. The degree of contribution of learning data is calculated using the difference,
14. The language model construction support method according to any one of supplementary notes 8 to 13, characterized in that:

(Appendix 15)
On the computer,
Processing to extract language features for each learning data,
A process of storing language feature origin information that associates the extracted language feature with the learning data that is its origin;
Processing for acquiring and storing language feature usage statistics information indicating the degree to which each language feature is used in speech recognition processing performed in the past;
A process for calculating the contribution of each learning data in the speech recognition process based on the language feature quantity statistical information and the language feature quantity origin information;
A process of outputting the calculated learning data contribution,
A program for running

(Appendix 16)
In the process of calculating the contribution degree, a value corresponding to the appearance degree of the language feature amount used in the speech recognition process in each learning data is calculated as the contribution degree.
The program according to supplementary note 15, characterized by:

(Appendix 17)
In the computer,
Based on at least one of correct text for the speech to be processed in the speech recognition process, information on the correctness of the speech recognition result of the speech, and correction information for the speech recognition result of the speech, the language feature usage statistics information To perform a process of weighting,
The language model construction support apparatus according to supplementary note 15 or 16, characterized in that:

(Appendix 18)
In the computer,
Based on the acquired language feature amount, a process for updating or constructing a language model used in the speech recognition process is executed.
18. The program according to any one of supplementary notes 15 to 17, characterized in that:

(Appendix 19)
In the computer,
Performing speech recognition on speech data using the language model, and further executing speech recognition processing for outputting the language feature usage statistics information;
19. The program according to any one of supplementary notes 15 to 18, characterized by:

(Appendix 20)
In the computer,
Causing at least one of learning data to be a contribution calculation target and language feature usage statistical information used for calculating the contribution to be switched or limited according to a user request,
20. The program according to any one of supplementary notes 15 to 19, characterized by:

(Appendix 21)
In the computer,
The difference between the linguistic feature quantity statistical information acquired in the speech recognition process performed using the language model before the update and the linguistic feature quantity statistical information acquired in the speech recognition process performed using the updated language model is obtained. , Execute the process of calculating the contribution of learning data using the difference,
21. The program according to any one of appendices 15 to 20, characterized in that:

  This application claims the priority on the basis of Japanese application Japanese Patent Application No. 2011-124785 for which it applied on June 3, 2011, and takes in those the indications of all here.

101, 201, 501 Language feature extraction unit 102, 202, 502 Language feature source storage unit 103, 203, 503 Language model learning unit 104, 204 Speech recognition unit 105, 205 Learning data contribution analysis unit

Claims (9)

Language feature extraction means for extracting a language feature for each learning data;
Means for storing language feature quantity origin information in which the extracted language feature quantity is associated with learning data that is the origin of the extracted language feature quantity;
Means for acquiring and storing linguistic feature quantity usage statistical information indicating the degree of use of each linguistic feature quantity in speech recognition processing performed in the past;
A contribution degree calculating means for calculating a contribution degree of each learning data in the speech recognition processing based on the language feature quantity statistical information and the language feature quantity origin information;
Means for outputting the calculated contribution of the learning data;
A language model construction support apparatus comprising: 2. The language model construction according to claim 1, wherein the contribution degree calculating unit calculates a value corresponding to the appearance degree of the language feature amount used in the speech recognition process in each learning data as a contribution degree. Support device. Based on at least one of correct text for the speech to be processed in the speech recognition process, information on the correctness of the speech recognition result of the speech, and correction information for the speech recognition result of the speech, the language feature usage statistics information Weight,
The language model construction support apparatus according to claim 1 or 2, characterized in that Updating or constructing a language model used in the speech recognition processing based on the language feature amount acquired by the language feature amount extraction unit;
The language model construction support apparatus according to any one of claims 1 to 3, wherein Further comprising speech recognition means for performing speech recognition on speech data using the language model and outputting the language feature usage statistics information;
The language model construction support apparatus according to claim 1, wherein the language model construction support apparatus is any one of claims 1 to 4. Switching or limiting at least one of the learning data to be calculated for contribution and the language feature usage statistical information used for calculating the contribution according to the user's request,
6. The language model construction support apparatus according to claim 1, wherein The difference between the linguistic feature quantity statistical information acquired in the speech recognition process performed using the language model before the update and the linguistic feature quantity statistical information acquired in the speech recognition process performed using the updated language model is obtained. The degree of contribution of learning data is calculated using the difference,
The language model construction support apparatus according to any one of claims 1 to 6, wherein A language model construction support method,
Extract language features from learning data,
Stores language feature origin information that associates the extracted language feature with the learning data that is its origin,
Language feature usage statistics information indicating the degree to which each language feature is used in the speech recognition process is acquired and stored,
Based on the linguistic feature quantity statistical information and the linguistic feature quantity origin information, the contribution of each learning data in the speech recognition processing is calculated,
Outputting the calculated contribution degree of the learning data;
A language model construction support method characterized by this. On the computer,
Processing to extract language features from learning data,
A process of storing language feature origin information that associates the extracted language feature with the learning data that is its origin;
Processing for acquiring and storing linguistic feature quantity usage statistical information indicating the degree to which each linguistic feature quantity is used in the speech recognition process;
A process for calculating the contribution of each learning data in the speech recognition process based on the language feature quantity statistical information and the language feature quantity origin information;
A process of outputting the calculated learning data contribution,
A program for running

Images