JP2017058534A - Language model creation device, language model creation method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make accuracy of language co-occur information high.SOLUTION: A map information storage unit 1 stores map information structured by associating a word representing a place name with geographical positional information. A language model storage unit 2 stores a language model for outputting prediction probability of an input word. A map information language model creation unit 3 uses the map information to create a map information language model for outputting a geographical distance between place names represented by two input words. A language model integration unit 5 creates an integrated language model formed by integrating the language model with the map information language model.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a technique for creating a language model used in natural language processing or the like.

  For example, speech recognition and machine translation require a language model for linguistic prediction. The language model is used to measure the goodness of linguistic connections, and its performance affects the performance of speech recognition and machine translation. So far, various kinds of language models have been proposed.

  A language model basically acquires linguistic connections from text. There are two main means of acquiring linguistic connections from text.

The first is a framework for acquiring linguistic connections from word chain information. In the case of the N-gram language model, which is a typical language model, it is examined how many times the frequency of the N word set appears on the text, and the immediately preceding N-1 word w _{i-N + 1} , ..., A linguistic connection is defined by constructing a prediction probability P (w _i | w _{i−N + 1} ,..., w _i−1 ) of the current word w _i from w _i−1 . The learning method of the N-gram language model is a known technique and is described in Non-Patent Document 1, for example.

The second is a framework for acquiring linguistic connections from word co-occurrence information. For example, in the trigger model, we investigate what word a word co-occurs in a document unit, and determine the probability P (w _b | w _a ) that a word w _b appears when _a word w _a appears. Model. Unlike the word chain, the model for word co-occurrence information can reflect long-distance information.

  In order to build a high-accuracy language model, these frameworks are used as representatives to combine various information obtained from text and use short-distance word chain information and long-distance word co-occurrence information. The goodness of social connections.

Kitakenji, “Language and Computation (4) Stochastic Language Model”, The University of Tokyo Press, November 1999, pp. 57-62

  A problem of the framework for acquiring existing word co-occurrence information is that a large-scale document set is required to acquire the co-occurrence information. The document needs to be a document suitable for a target task (for example, for contact center speech recognition, patent translation, etc.). In many cases, a large amount of documents suitable for a task cannot be collected. If the amount of co-occurrence information that can be acquired decreases, the language model lacks the robustness of language constraints. This problem is generally called the data sparseness problem in language models.

  In view of the above, an object of the present invention is to provide a language model creation technique capable of improving the accuracy of language co-occurrence information.

  In order to solve the above-described problem, a language model creation device according to the present invention is inputted with a map information storage unit that stores map information structured by associating a word representing a place name with geographical position information, A map that generates a map information language model that outputs a geographical distance between place names represented by two input words, using a language model storage unit that stores a language model that outputs a word prediction probability, and map information An information language model generation unit; and a language model integration unit that generates an integrated language model obtained by integrating the language model and the map information language model.

  According to the language model creation technique of the present invention, since the prediction probability is calculated in consideration of the geographical distance for the word representing the place name, the language co-occurrence information can be highly accurate.

FIG. 1 is a diagram illustrating a functional configuration of a language model creation device. FIG. 2 is a diagram illustrating a processing procedure of the language model creation method.

  The present invention is a technique for constructing a language model that promotes linguistic prediction by using distance information between place names on a map. To solve the problem of data sparseness when modeling the co-occurrence information described above, for example, a search system such as a car navigation system or a local information search system can be used to focus on tasks where place names are important and present a solution. it can. Specifically, structured information called maps is introduced into language prediction of language models. Distance information between place names can be acquired from the map information. And this information is already structured in Japan and around the world. For example, in a system included in a car navigation system, such map information is structured and can be easily extracted.

  Explain how distance information between place names is useful for language models. It is considered that place names that are close to each other can be constrained to appear relatively easily in the same conversation. For example, if a topic related to “Shinjuku” is currently being discussed, words such as “Shibuya” and “Ikebukuro” that represent place names that are close in distance are likely to rise to the topic, such as “Wakkanai” and “Naha”. Words representing place names that are far away are less likely to be talked about. It is considered that the data sparseness of the language model can be solved by introducing such language constraints between various place names.

  Hereinafter, embodiments of the present invention will be described in detail. In addition, the same number is attached | subjected to the component which has the same function in drawing, and duplication description is abbreviate | omitted.

  As shown in FIG. 1, the language model creation device of the embodiment includes a map information storage unit 1, a language model storage unit 2, a map information language model generation unit 3, a map information language model storage unit 4, a language model integration unit 5, And an integrated language model storage unit 6. The language model creation method of the embodiment is realized by the processing of each step described later by this language model creation device.

  The language model creation device is, for example, a special program configured by reading a special program into a known or dedicated computer having a central processing unit (CPU), a main memory (RAM), and the like. Device. For example, the language model creation device executes each process under the control of the central processing unit. Data input to the language model creation device and data obtained in each process are stored in the main storage device, for example, and the data stored in the main storage device is read out as necessary and used for other processing. Is done. In addition, at least a part of each processing unit of the language model creation device may be configured by hardware such as an integrated circuit. Each storage unit of the language model creation device is, for example, a main storage device such as a RAM (Random Access Memory), an auxiliary storage device configured by a semiconductor memory element such as a hard disk, an optical disk, or a flash memory, or a relational It can be configured with middleware such as a database or key-value store.

  The map information storage unit 1 stores structured map information. The structured map information is data in which names of places are digitized, and is data that can acquire the distance relationship between place names and addresses of place names. Specifically, it is information in which a word representing a place name is associated with geographical location information of the place name, and the coordinates or address of the designated place name can be obtained. For example, in a car navigation system, structured map information that satisfies this condition is used. In another example, map information such as “Google (registered trademark) map” available on the Web corresponds to structured map information.

  The language model storage unit 2 stores a normal language model. The normal language model is a language model that has been used conventionally, and is a model that outputs the prediction probability of an input word. In this embodiment, an N-gram language model will be described as an example.

  With reference to FIG. 2, the processing procedure of the language model creation method of the embodiment will be described.

  In step S1, the map information language model generation unit 3 uses the map information stored in the map information storage unit 1 as input to input a word representing two place names, and outputs a geographical distance between the place names. Generate an information language model. The generated map information language model is stored in the map information language model storage unit 4.

Hereinafter, a method for creating a map information language model will be described in detail. First, place names of all places where position information can be acquired in the structured map information are listed. This is, for example, a place name such as “Tokyo Tower” or “Fuji Television”. If the task of speech recognition or machine translation can be narrowed down in advance, the geographical range for acquiring the place name may be limited, such as “within Tokyo” or “within Niigata City”. Here, it is assumed that there are M listed place names. Next, for the M place names listed, distances are calculated from structured map information for all two pairs. Specifically, the distance between the two points is calculated based on the coordinate information of the two selected points. Since the method for calculating the distance from the latitude and longitude of the two points is well known, detailed description is omitted here. When the distance between all two points is calculated, M * (M-1) * 1/2 distances are finally obtained. Information about this distance becomes a map information language model. That is, the geographical information d (w _a , w _b ) between the name w _{a of a} point and the name w _b of _a point is the map information language model.

  In step S2, the language model integration unit 5 integrates the normal language model stored in the language model storage unit 2 and the map information language model stored in the map information language model storage unit 4 to obtain an integrated language model. Is generated. The generated integrated language model is stored in the integrated language model storage unit 6.

Hereinafter, a method for creating an integrated language model will be described in detail. In the N-gram language model, the prediction probability P (w _i | w _{i-N + 1} , ..., w of the current word w _i from the immediately preceding N-1 word w _{i-N + 1} , ..., w _{i-1. i-1} ). In order to integrate the map information language model, context constraints are required. The context constraint is the immediately preceding N-1 word in the N-gram language model. The context constraints of the map information language model can be given in various ways. Here, for the sake of simplicity, the context of w _i is assumed to be h _i . For example, if the 20 words of the immediately preceding context constraints of map information language model, h _i is w _i-21, ..., given the w _i-1. If the score obtained from the map information language model at this time is d (w _i | h _i ), the integrated language model S (w _i | h _i ) is given by the following equation.

Here, α is an integrated weight. The integration weight is given manually. The integration weight may be set so that the performance of speech recognition and machine translation is high, but can be set to 0.0001, for example.

An actual method for obtaining d (w _i | h _i ) will be described. As in the previous example, the context constraint of the map information language model is the previous 20 words. At this time, d (w _i | h _i ) is given by the following equation.

That is, the values modeled by the map information language model are obtained for all the words in the context constraint, and the values are summed. If w _i is not included in the M word in the map information language model, d (w _i | h _i ) is 0. If w _k is not included in the M word in the map information language model, d (w _i , w _k ) becomes infinite. By configuring in this way, a robust score can be given to the M words in the map information language model. As a result, performance improvement can be expected by using this language model for speech recognition and machine translation.

  By configuring as described above, the language model creation technique of the present invention can construct a language model in which language constraints using distance information between place names on a map are introduced. Thereby, the robustness of the language restriction regarding word co-occurrence can be increased, and the problem of data sparseness can be avoided. As a result, this language model can be used in tasks where place names are important, such as car navigation systems and local information retrieval systems, so that speech recognition can achieve higher recognition performance than before. Provides better translation performance than ever.

  The present invention is not limited to the above-described embodiment, and it goes without saying that modifications can be made as appropriate without departing from the spirit of the present invention. The various processes described in the above embodiment may be executed not only in time series according to the order of description, but also in parallel or individually as required by the processing capability of the apparatus that executes the processes or as necessary.

[Program, recording medium]
When various processing functions in each device described in the above embodiment are realized by a computer, the processing contents of the functions that each device should have are described by a program. Then, by executing this program on a computer, various processing functions in each of the above devices are realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used.

  This program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. When executing the process, the computer reads a program stored in its own recording medium and executes a process according to the read program. As another execution form of the program, the computer may directly read the program from a portable recording medium and execute processing according to the program, and the program is transferred from the server computer to the computer. Each time, the processing according to the received program may be executed sequentially. A configuration in which the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes a processing function only by an execution instruction and result acquisition without transferring a program from the server computer to the computer. It is good. Note that the program in this embodiment includes information that is used for processing by an electronic computer and that conforms to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer).

  In this embodiment, the present apparatus is configured by executing a predetermined program on a computer. However, at least a part of these processing contents may be realized by hardware.

1 Map information storage unit 2 Language model storage unit 3 Map information language model generation unit 4 Map information language model storage unit 5 Language model integration unit 6 Integrated language model storage unit

Claims (4)

A map information storage unit for storing map information structured by associating a word representing a place name with geographical location information;
A language model storage unit that stores a language model that outputs a predicted probability of the input word;
A map information language model generating unit that generates a map information language model that outputs a geographical distance between the place names represented by the two input words using the map information;
A language model integration unit for generating an integrated language model obtained by integrating the language model and the map information language model;
Language model creation device including The language model creation device according to claim 1,
The above language model is based on the prediction probability P (w _i | w _{i-N + 1} , ..., w _{i- of the} current word w _i from the previous N-1 word w _{i-N + 1} , ..., w _{i-1. 1} ) N-gram language model for
The map information language model is a language model for obtaining a geographical distance d (w _a , w _b ) between a word w _a representing _a place name and a word w _b representing _a place name,
In the integrated language model, h _i is a context constraint including M−1 words w _{i−M + 1} ,..., W _i−1 , and d (w _i | h _i ) is a current word w _i and a context. The sum of the geographical distances from the words w _{i-M + 1} ,..., W _i-1 included in the constraint h _i , α is a predetermined integrated weight, and is expressed by the following equation:

Language model creation device. The map information storage unit stores map information that is structured by associating words representing place names with geographical location information,
The language model storage unit stores a language model that outputs the predicted probability of the input word,
A map information language model generation unit that generates a map information language model that outputs a geographical distance between place names represented by the two input words, using the map information;
A language model integration unit for generating an integrated language model in which the language model integration unit integrates the language model and the map information language model;
Language model creation method including   A program for causing a computer to function as the language model creation device according to claim 1.

Images