JP2016180849A - Learning data generation unit, language model learning unit, learning data generation method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a learning data generation unit capable of generating many sets of n words which are natural as neighboring languages from a piece of given text data while preventing generation of any sets of n words which are never neighboring each other.SOLUTION: The learning data generation unit includes: a tree structure generation section that reads a predetermined length of a text data in which each of words is segmented and a piece of data on dependency between clause phrases in the text data to generate a tree structure which represents a dependency relation among the text data in the predetermined length; and a generate text generation section that generates a new text after removing a part or all of clause phrases which have no dependency to other clause phrase based on the tree structure.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a learning data generation device that generates learning data of a statistical language model used in speech recognition, statistical translation, natural language processing, and the like, a learning data generation method, a language model learning device that learns a statistical language model, and a program About.

  Learning a statistical language model such as a word n-gram requires a large amount of sentence data, but even if the amount of sentence data increases, only a part of all n sets of words can be obtained. Always occurs. For this reason, it is necessary to devise ways to use text data as much as possible.

  For example, in Non-Patent Document 1, an attempt is made to generate n sets of words obtained by skipping words in between without necessarily being connected, and to use this as learning data for the word n-gram. .

Guthrie, David, et al., "A closer look at skip-gram modeling", Proceedings of the 5th International Conference on Language Resources and Evaluation (LREC). 2006.

  When learning data is generated by the method of Non-Patent Document 1, there is a risk of generating up to n sets of words that are linguistically adjacent to each other. Even if such learning data is used, high-precision statistical data may be generated. I could not learn a language model.

  SUMMARY OF THE INVENTION Accordingly, an object of the present invention is to provide a learning data generation device that can suppress the generation of n adjacent words that cannot be adjacent to each other and can generate many natural words as a language that can be adjacent from given sentence data. To do.

  The learning data generation apparatus of the present invention includes a tree structure generation unit and a sentence generation unit.

  The tree structure generation unit reads the dependency data between the sentence data divided by words and the clauses of the sentence data with a predetermined length, and generates a tree structure representing the dependency relation of the sentence data with a predetermined length. . The sentence generation unit generates a new sentence by removing some or all of the phrases that have not received other phrases based on the tree structure.

  According to the learning data generation apparatus of the present invention, generation of n sets of words that cannot be adjacent to each other can be suppressed, and many natural word sets as words that can be adjacent to each other can be generated from given sentence data.

1 is a block diagram illustrating a configuration of a learning data generation device and a language model learning device according to Embodiment 1. FIG. 3 is a flowchart illustrating operations of the learning data generation device and the language model learning device according to the first embodiment. The figure which illustrates the tree structure which a tree structure generation part generates.

  Hereinafter, embodiments of the present invention will be described in detail. In addition, the same number is attached | subjected to the structure part which has the same function, and duplication description is abbreviate | omitted.

  The configuration and operation of the learning data generation device and the language model learning device according to this embodiment will be described with reference to FIGS. FIG. 1 is a block diagram illustrating a configuration of a learning data generation device 1 and a language model learning device 2 according to the present embodiment. FIG. 2 is a flowchart illustrating operations of the learning data generation device 1 and the language model learning device 2 according to this embodiment.

  As shown in FIG. 1, the learning data generation apparatus 1 of the present embodiment includes a tree structure generation unit 11 and a sentence generation unit 12. Moreover, in addition to the function of the learning data generation device 1, it is good also as the language model learning device 2 which performs even the production | generation of a statistical language model. In this case, the language model learning device 2 is configured to further include a language model learning unit 21 in addition to the configuration of the learning data generation device 1.

  Assume that a text data storage unit 8 and a dependency data storage unit 9 exist as storage areas outside the learning data generation device 1 and the language model learning device 2. The text data storage unit 8 and the dependency data storage unit 9 may be provided inside the learning data generation device 1 and the language model learning device 2. It is assumed that the sentence data storage unit 8 stores sentence data divided for each word. It is assumed that the dependency data storage unit 9 stores dependency data between phrases of the sentence data stored in the sentence data storage unit 8.

Sentence data divided into words (also referred to as word string data) can be obtained by morphological analysis. The morpheme analysis can be performed, for example, by the method described in Reference Non-Patent Document 1.
(Reference Non-Patent Document 1: Yuji Matsumoto, “A morphological analysis system“ tea bowl ””, Information Processing, vol. 41 (11), pp. 1208-1214, 2000)

Dependency analysis is a technology that uses a morpheme analysis result as an input to construct a phrase composed of a plurality of words and predicts the dependency relation between phrases based on the relationship between the part of speech, the appearance form of the word, and the ID of the word. . The dependency analysis can be performed by a method described in Reference Non-Patent Document 2, for example.
(Reference Non-patent Document 2: Taku Kudo, Yuji Matsumoto, “Japanese Dependency Analysis by Chunking Stage Application”, IPSJ Transactions, 43 (6), pp. 1834-1842, 2002)

  The tree structure generation unit 11 reads dependency data between sentence data divided into words and phrases of sentence data by a predetermined length (for example, one sentence at a time), and determines dependency relations of sentence data of a predetermined length. A tree structure to represent is generated (S11). Based on the tree structure, the sentence generation unit 12 generates a new sentence by removing some or all of the phrases that have not received other phrases (as a dependency) (S12). The language model learning unit 21 learns a language model using the original sentence and the plurality of generated sentences (S21).

  Hereinafter, the above-described steps S11 and S12 will be described in detail with reference to the example of FIG. FIG. 3 is a diagram illustrating a tree structure generated by the tree structure generation unit 11.

  The dependency relationship is information indicating which clause a certain clause relates to. The dependency relationship is indicated by the clause number of the source and the clause number of the destination. Thereby, the tree structure generation unit 11 generates a tree structure as shown in FIG. 3, for example (S11). The tree structure generation unit 11 records the word numbers belonging to the phrase, thereby associating the structure inside the phrase with the tree.

The sentence generation unit 12 can generate various sentences by removing a phrase that has not received any phrase (as a dependency) from the root of the tree. In addition, even if a clause is affected by another clause, a clause that starts with an auxiliary word can be removed as an unrelated clause including that clause. For example, in FIG. 3, the phrases “every”, “all”, “my”, and “twist” (the phrase that has been dot-hatched in FIG. 3) have not received any of the phrases. The sentence generation unit 12 removes these clauses at random, which is the original sentence data “twisting all the reality to yourself”.
From the new sentence, "Bent all reality",
“We twisted every reality.”
“I twisted all reality to myself.”
“I bent all the reality”
Etc. are generated (S12). Of the above four examples, the other than the third example is an example in which the two phrases “toward yourself” including the phrase starting from the auxiliary word are removed. As a result, the language model learning unit 21 sets the new word triplet “reality- + bend” “reality- + twist” “reality- + self”
A skip-gram which can be naturally obtained can be obtained. The distance to be skipped is not explicitly set as a fixed value as in the conventional case, so the case of the above three natural words triple set such as 6, 5, or 2 skipped as in the above case. Many can be obtained.

The principle of generating a new sentence is to remove some or all of the phrases that have not received any of the phrases. If there are n clauses that have not received any clauses, a total of 2 ⁿ new sentences can be generated from the removal of one clause to the removal of all n clauses. The language model learning unit 21 may not remove all the clauses that have received any of the clauses equally but may remove these clauses in a probabilistic manner.

  According to the learning data generation device 1 and the language model learning device 2 of the present embodiment, the learning data that can be a natural sentence based on the dependency information of the sentence data in a skipping method different from the conventional skip-gram. Since a set of n words can be created, it is possible to acquire a larger number of n sets of words that are more natural than before from the same given sentence data and use it for language model learning. In addition, the learning data generated by the learning data generation apparatus 1 according to the present embodiment is not only the word Ngram, but also the part of speech Ngram that can be created from the part of speech corresponding to the obtained word is natural. This data can also be used for model learning.

<Supplementary note>
The apparatus of the present invention includes, for example, a single hardware entity as an input unit to which a keyboard or the like can be connected, an output unit to which a liquid crystal display or the like can be connected, and a communication device (for example, a communication cable) capable of communicating outside the hardware entity. Can be connected to a communication unit, a CPU (Central Processing Unit, may include a cache memory or a register), a RAM or ROM that is a memory, an external storage device that is a hard disk, and an input unit, an output unit, or a communication unit thereof , A CPU, a RAM, a ROM, and a bus connected so that data can be exchanged between the external storage devices. If necessary, the hardware entity may be provided with a device (drive) that can read and write a recording medium such as a CD-ROM. A physical entity having such hardware resources includes a general-purpose computer.

  The external storage device of the hardware entity stores a program necessary for realizing the above functions and data necessary for processing the program (not limited to the external storage device, for example, reading a program) It may be stored in a ROM that is a dedicated storage device). Data obtained by the processing of these programs is appropriately stored in a RAM or an external storage device.

  In the hardware entity, each program stored in an external storage device (or ROM or the like) and data necessary for processing each program are read into a memory as necessary, and are interpreted and executed by a CPU as appropriate. . As a result, the CPU realizes a predetermined function (respective component requirements expressed as the above-described unit, unit, etc.).

  The present invention is not limited to the above-described embodiment, and can be appropriately changed without departing from the spirit of the present invention. In addition, the processing described in the above embodiment may be executed not only in time series according to the order of description but also in parallel or individually as required by the processing capability of the apparatus that executes the processing. .

  As described above, when the processing functions in the hardware entity (the apparatus of the present invention) described in the above embodiments are realized by a computer, the processing contents of the functions that the hardware entity should have are described by a program. Then, by executing this program on a computer, the processing functions in the hardware entity are realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like, and as an optical disk, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only). Memory), CD-R (Recordable) / RW (ReWritable), etc., magneto-optical recording medium, MO (Magneto-Optical disc), etc., semiconductor memory, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. Can be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. When executing the process, the computer reads a program stored in its own recording medium and executes a process according to the read program. As another execution form of the program, the computer may directly read the program from a portable recording medium and execute processing according to the program, and the program is transferred from the server computer to the computer. Each time, the processing according to the received program may be executed sequentially. Also, the program is not transferred from the server computer to the computer, and the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by the execution instruction and result acquisition. It is good. Note that the program in this embodiment includes information that is used for processing by an electronic computer and that conforms to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer).

  In this embodiment, a hardware entity is configured by executing a predetermined program on a computer. However, at least a part of these processing contents may be realized by hardware.

Claims (5)

A tree structure generation unit that reads sentence data divided for each word and dependency data between clauses of the sentence data by a predetermined length, and generates a tree structure representing the dependency relation of the sentence data of the predetermined length When,
A learning data generation device including a sentence generation unit that generates a new sentence by removing a part or all of a phrase that has not received another phrase based on the tree structure. A tree structure generation unit that reads sentence data divided for each word and dependency data between clauses of the sentence data by a predetermined length, and generates a tree structure representing the dependency relation of the sentence data of the predetermined length When,
Based on the tree structure, a sentence generation unit that generates a new sentence by removing some or all of the phrases that have not received other phrases;
A language model learning apparatus including a language model learning unit that learns a language model using the generated sentence. A learning data generation method executed by a learning data generation device,
The learning data generation device includes:
Reading dependency data between sentence data divided into words and clauses of the sentence data with a predetermined length, and generating a tree structure representing a dependency relation of the sentence data with the predetermined length;
A learning data generation method for executing a step of generating a new sentence by removing a part or all of a phrase that has not received another phrase based on the tree structure.   A program that causes a computer to function as the learning data generation device according to claim 1.   A program for causing a computer to function as the language model learning device according to claim 2.

Images