JP2011033806A - Language model compression device, access device of language model, language model compression method, access method of language model, language model compression program, and access program of language model - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide technology of efficiently accessing a language model by suppressing the amount of data of an n-gram language model as a learning result. <P>SOLUTION: The language model compression device 1 stores the n-gram language model in a language model storage section 5. A data structure conversion section 3 converts a pointer for indicating a first position of an (n+1)-gram in a data arrangement of the n-gram language model stored in the language model storage section 5, to fixed byte expression, and stores it in a conversion data storage section 6. A compression section 4 of pointer expression makes a virtual trie structure by providing a virtual route node in a tree structure of the n-gram language model stored in a conversion data storage section 6, and the pointer is compressed and converted to a level-order unary degree sequence (LOUDS) expression. The compressed and converted data is stored in a compression data storage section 7. A storage device (RAM) of a computer is mainly used for the storage section 7. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention divides a word into units of n characters, and a technique for compressing an n-gram language model obtained from each word string and its appearance frequency (probability), and efficiently accessing the n-gram language model It relates to technology.

  As is well known, in the field of speech recognition and statistical machine translation, as shown in Non-Patent Document 1, in order to model the likelihood of an output language (word string), a word consisting of n words The conditional probability (Equation 1) for a sequence (called n-gram) is widely used.

  The likelihood (Formula 3) of the word string (Formula 2) can be approximately calculated as Formula 4 using the conditional probability of Formula 1.

Such a model that gives the conditional probability of Equation 1 is called an n-gram language model. As the magnitude of “n”, a value of “3-4” is often used for speech recognition, and a value of “4-5” is often used for statistical machine translation. The conditional probability of Equation 1 is recursively calculated as Equation 5. In addition, Equation 2, the words w _1, w _2, ···, is obtained by omitting the title the w _n.

  In order to use such a statistical model in a computer (computer), first, information such as the smoothed probability of Expression 6 and the backoff coefficient of Expression 7 according to the word string (Expression 2) appearing in the learning data. , And the word string (Equation 2) as an input must be able to be accessed efficiently.

  In order to improve the performance of n-gram applications such as speech recognition and statistical machine translation, as shown in Non-Patent Document 2, not only learning these parameters using a large amount of learning data, but also learning It is known that by increasing the data, it is effective to maintain a large number of variations (n-gram variations) of the correct word string (formula 2) appearing there.

  Further, it is known that the n-gram language model can be expressed by a tree structure (data structure) as shown in Non-Patent Document 3. Furthermore, as shown in Non-Patent Document 4.5, LOUDS (Level-Order Undegree Sequence Sequence) is known as a technique for compactly expressing a trie that is a kind of tree structure. Here, a trie is a kind of an ordered tree structure having one root node, and is also called a prefix tree (Prefix Tree).

S. M.M. Katz, “Estimation of Probabilities from Sparse Data for the Language Model Component of a Speech Recognizer,” IEEE TRANSACTIONS ON ACUUS. ASSP-35, NO. 3, MARCH 1987, pp. 400-401 T.A. Brants, A.B. C. Popat, P.M. Xu, F.A. J. et al. Och, and J.M. Dean, Large Language Models in Machine Translation, Proceedings of the 2007, Joint Conferencing on Empirical Principles in Natural Language Processing. 858--867, 2007. B. Raj and E.M. W. D. Whitaker. “LOSSLESS COMPRESION OF LANGUAGE MODEL STRUCTURE AND WORD IDENTIFIERS” In Proceedings of ICASSP, volume 1, pages I≡388≡I≡391 vol. 1, April 2003. O'Neil Delpratt, Naila Rahman, and Rajeev Raman. "Engineering the LOUDS Success Tree Representation" In Proceedings of the 5th International Workshop on Exponential Algorithms, pages 134≡145, 2006. Guy Jacobson. “Space-Efficient Static Trees and Graphs” In 30th Annual Symposium on Foundations of Computer Science, pages 549≡554, Nov 1989. Dong Kyue Kim, Joong Chae Na, Ji Eun Kim, and Kunsoo Park. “Efficient implementation of rank and select functions for successful re-presentation”, In Proceedings of the 5th International Working on Exponential 3

  However, since the n-gram language model learned from a large amount of data stores a very wide variety of n-grams, the model expression becomes huge. In particular, since efficient access is required when using the n-gram language model, the data structure considering it tends to be large.

  On the other hand, in order to achieve efficient access, it is preferable to store the n-gram language model in the main memory (RAM), but even with a modern computer (computer), the storage capacity of the main memory Has its limits. For example, if a conventional method is used to express a huge n-gram learned from all Web data, a capacity of 40 GB or more is required. Therefore, it is difficult to store the huge n-gram in the main storage device except for some very expensive computers. It is.

  In addition, the tree structure representation of the n-gram language model has no root node (or not one). Although a method for expressing the trie structure in a compact manner is known, since the data structure of the n-gram language model is not a trie, the method for expressing the trie structure in a compact manner can be applied to the representation of the n-gram language model. Further ingenuity is necessary.

  The present invention has been made in order to solve the above-described problems of the prior art, and provides a technology that can efficiently access data by suppressing the data amount of the n-gram language model. And

  Therefore, the present invention converts the data structure of an n-gram language model used in a natural language processing system or the like into a trie (tree structure) and expresses the trie with a smaller number of bit strings by a technique called LOUDS. Furthermore, this method is improved in consideration of the characteristics of n-gram.

  One aspect of the language model compression apparatus according to the present invention is provided with a virtual root node, a data structure conversion unit that converts the structure of an n-gram language model into a trie structure, and the data structure conversion unit Compression means for compressing and converting the trie structure into a LOUDS (Level-Order Unary Degree Sequence) expression.

In another aspect of the language model according to the present invention, the position (node ID) of the first node of the highest order of the n-gram language model is stored in a storage device, and a pointer expression representing the structure of the n-gram language model is stored. Data structure conversion means for converting the highest-order n-gram into a deleted expression is provided. Here, the number N ₁ of 1-grams is stored in the storage device, and the structure of the n-gram language model converted by the data structure conversion means is a super node (virtual root node indicating the root node of the tri structure). And compression means for compressing and converting the data into a LOUDS expression expanded so as not to have a bit corresponding to the highest order n-gram.

In one aspect of the language model access device according to the present invention, the position of the parent node “(n−1) -gram” (parent) is expressed by Expression 8 with respect to the n-gram position x of the input word string. First access means for calculating and outputting by
The position of the child node (first_child) having the smallest word ID among the child nodes “(n + 1) -gram” is calculated by Expression 9 with respect to the n-gram position x of the input word string and output. Two access means.

  According to another aspect of the language model access device of the present invention, the position of the parent node “(n−1) -gram” (parent) is expressed with respect to the position x of the n-gram of the input word string. The first access means calculated and output by 10 and the child node (first_child) having the smallest word ID among the child nodes “(n + 1) -gram” with respect to the n-gram position x of the input word string The second access means for calculating and outputting the position of

  One aspect of the language model compression method according to the present invention is a data structure conversion step in which the data structure conversion means provides a virtual root node to convert the structure of the n-gram language model into a trie structure, and the compression means And a compression step of compressing and converting the trie structure converted in the data structure conversion step into a LOUDS (Level-Order Unary Degree Sequence) expression.

In another aspect of the language model compression method according to the present invention, the data structure conversion means stores the position (node ID) of the first node of the highest degree of the n-gram language model in the storage device, and the n-gram language A data structure conversion step of converting the pointer expression representing the structure of the model into an expression from which the highest order n-gram is deleted. Here, the compression means stores the number N ₁ of 1-grams in a storage device, and the structure of the n-gram language model converted by the data structure conversion means is a super node (virtual node indicating the root node of the tri structure). And a compression step of compressing and converting the data into a LOUDS expression expanded so as not to have a bit corresponding to the highest order n-gram.

  In one aspect of the language model access method according to the present invention, the first access means performs the parent node “(n−1) -gram” (parent) with respect to the n-gram position x of the input word string. ) And calculating and outputting the position of () by the expression 8, and the second access means includes the child node “(n + 1) -gram” with respect to the position x of the n-gram of the input word string. And calculating and outputting the position of the child node (first_child) having the smallest word ID according to equation (9).

  In another aspect of the language model access method according to the present invention, the first access means performs a parent node “(n−1) -gram” (with respect to the n-gram position x of the input word string. The position of parent) is calculated and output by Expression 10, and the second access means sets the child node “(n + 1) -gram” to the position x of the n-gram of the input word string. A step of calculating and outputting the position of the child node (first_child) having the smallest word ID according to equation (11).

  In addition, this invention is good also as an aspect of the program for functioning a computer as said each apparatus. This program can be distributed and provided in a form recorded on a recording medium.

  According to the present invention, the data amount of the pointer expression that defines the structure of the n-gram language model is suppressed. As a result, even with a huge n-gram language model, it is possible to hold all or most of the pointer representation on a high-speed memory (RAM) instead of a hard-disk device that is slow to access. Become. The present invention makes it possible to efficiently access an n-gram language model in machine translation and speech recognition using a huge n-gram language model, which contributes to speeding up of processing.

The lineblock diagram of the language model compression device and language model access device concerning an embodiment of the present invention. The data structure figure of the n-gram language model memorize | stored in the same language model memory | storage part. The data structure figure of the n-gram language model after the conversion of the data structure conversion part. The figure which shows the pointer expression based on a trie structure. The figure which shows the pointer expression based on an n-gram structure. Structure diagram of the trie structure. Structure diagram of n-gram.

  Hereinafter, a language model compression device and a language model access device according to an embodiment of the present invention will be described. Both of these devices are used as a combination in a natural language processing system such as a speech recognition device or a machine translator. Here, the compression device suppresses the data amount of the n-gram language model generated by the n-gram learning device, while the access device suppresses the data capacity by the compression device for speech recognition and statistical machine translation. Access the generated n-gram language model.

  Specifically, each of the devices is configured by a computer and includes hardware resources of a normal computer, for example, a main storage device such as a CPU and a memory (RAM), a hard disk drive device, and the like.

  As a result of the cooperation between the hardware resource and the software resource (OS, application, etc.), as shown in FIG. 1, the compression apparatus 1 includes a data structure conversion unit 3, a pointer expression compression unit 4, a language model storage unit. 5, the conversion data storage unit 6 and the compressed data storage unit 7 are mounted, while the access device 2 mounts an n-gram access unit (first_child) 8 and an n-gram access unit (parent) 9. Both devices 1.2 are not necessarily composed of a plurality of computers, and may be composed of a single computer.

  In brief, each of the storage units 5 to 7 is constructed in the hard disk drive device or the main storage device. Among these, the language model storage unit 5 stores a language model generated as a learning result based on a large amount of learning data (for example, Web data). It is assumed that the language model stored here is expressed in the normal “data structure of n-gram language model” format.

  As shown by an arrow A, the data structure conversion unit 3 receives the language model of the language model storage unit 5 and converts it into an n-gram language model data structure represented by a fixed byte of a pointer (data conversion step). ). The converted data is stored in the converted data storage unit 6 as indicated by an arrow B.

  As indicated by an arrow C, the pointer representation compression unit 4 receives the accumulated data of the converted data storage unit 6 as an input, adds a process for compactly representing the pointer array to the accumulated data, and compresses the data amount ( Pointer expression compression step). That is, the data is compressed and converted into a data structure of an n-gram language model in which pointers are expressed in LOUDS, and stored in the compressed data storage unit 7 as indicated by an arrow D.

  The access device 2 receives a word string (n-gram) given through an input unit (not shown) as an input, As shown in F, the stored data of the compressed data storage unit 7, that is, the pointer of the compressed n-gram language model is accessed (access step). Specifically, the access unit 8 receives the position of an n-gram (word string) as indicated by an arrow G, and refers to the compressed data storage unit 7 as indicated by an arrow H ( The position of (n + 1) -gram is output. On the other hand, the access unit 9 receives the position of n-gram (word string) as indicated by an arrow I and refers to the compressed data storage unit 7 as indicated by an arrow J (n−1). ) -Gram output position. Hereinafter, the details of the functional blocks 3 to 9 of both the devices 1.2 will be described.

<< Language Model Compression Device 1 >>
(1) Data structure conversion unit 3
First, data stored in the language model storage unit 5, that is, an n-gram data structure will be described. This n-gram is expressed by the data structure of FIG. 2 (see Non-Patent Document 2), and “1-gram, 2-gram, 3-gram,...” Is a separate table of the language model storage unit 5. Expressed.

Each column of the table in the n-gram, the word w _n word ID of (word id), smoothed probability value of formula 6 (probability), the back-off factor of the formula 7 (back-off), ( X + 1) -It has a pointer indicating the first position of gram (X is an integer of 1 ≦ X ≦ n). Each table is represented as an array of such quadruples. An index of this array can be used as a pointer. For example, a pointer can be realized by a 4-byte integer.

Here, in the previous continuous area pointed to by the pointer of X-gram (X is an integer of 1 ≦ X ≦ n), (X + 1) -gram sharing the history of Expression 12 is in the order of word IDs of w _{x + 1.} Sorted and stored.

  The boundary of the end of this area is defined by the destination pointed by the pointer of the next entry of the X-gram. In order to search for a certain X-gram, a binary search is performed for each word constituting the X-gram, so a total of X binary searches are performed. By defining the word ID at the position of each row of the 1-gram table, the column of the word ID of 1-gram can be omitted as shown in FIG.

  The data structure conversion unit 3 converts the data structure of FIG. 2 stored in the language model storage unit 5, and as shown in FIG. 3, a word ID, a smoothed probability value, a backoff coefficient, a pointer Are represented in separate arrays and stored in the converted data storage unit 6. Here, 2-gram information is connected after 1-gram, 3-gram information is connected after 2-gram, and the connection processing is sequentially performed. The order boundaries (start positions of 2-gram, 3-gram,...) Are separately stored in the main storage device so that information on each order can be accessed.

  As the pointer array, either an expression based on the trie structure of FIG. 4 (first form) or an expression based on the n-gram structure of FIG. 5 (second form) is used. FIG. 4.5 shows a pointer structure corresponding to the tree structure of FIG. 6.7. It is assumed that the pointer that is an element of the array is expressed by a fixed byte (for example, 4 bytes) integer.

  In the second form, the position of the first entry of the highest order of the n-gram language model is stored, and the pointer of the highest order n-gram stored after that in the pointer array of the first form. Are not stored in the second form. However, it is only the pointer array that does not store the information of the highest order n-gram, the other word ID (word id), the smoothed probability value (probability) of Equation 6, and the back-off coefficient of Equation 7 ( For back-off), information on the highest order n-gram is also stored.

(2) Compression unit 4 for pointer expression
The compression unit 4 refers to the converted data storage unit 6, and compactly represents an array indicating a pointer among the arrays converted by the data structure conversion unit 3, and stores it in the converted data storage unit 6. To do. Hereinafter, the specific content of each form is demonstrated.

<First form>
First, the 1st form in the said compression part 4 is demonstrated. A trie is a kind of tree structure and is represented by a tree structure having one root node. The n-gram data structure shown in FIG. 2 is not a trie because there is no single root node, but here a virtual root node is provided to simulate the n-gram data structure as a trie. The pointer array is compressed by applying LOUDS (see Non-Patent Document 4.5), which is a compact trie expression method.

More specifically, in the LOUDS expression, the node IDs are assigned to the nodes in the hierarchical order of “1-gram, 2-gram,...” From left to right, that is, in order of width priority. Here, a node (parent node) having d (d ≧ 0) children (child nodes) is represented by a bit string of “1 ^d 0” (“1 ^d ” includes ^d “1”). Represents a bit string.) A super root node is provided as a further virtual node indicating a virtual root node. Since the child of the super root node is one root node, the super root node is represented by “10”.

  FIG. 6 shows an example of the trie structure. Table 1 shows pointer representations of the LOUDS bit string of this trie structure example.

According to the trie structure example, since the root node “0” has four children, the root node “0” is represented by four “1” s and one “0” representing the end. Here, a trie having M nodes is represented by (M + 1) “0” s and M “1s”, and thus is represented by a total of 2M + 1 bits.
If the number of X-grams is expressed as N _x and Expression 13 is used, Expression 14 is established in the n-gram language model. Therefore, in the n-gram language model, a pointer is expressed by the number of bits of Expression 15. Here, the pointer expressed in a compressed manner is stored in the compressed data storage unit 7.

In order to access the n-gram language model expressed in a compressed manner as described above, an operation called “select ₁ (i)” that returns the position of the i-th “1” in the bit string is defined on the LOUDS bit string. . Here, it is assumed that the bit positions start from “0” as shown in Table 1. Similarly, “select ₀ (i)” can be defined as an operation for returning the i-th “0” position in the bit string.

“Select _b (i) (b = 0 or 1)” can be efficiently realized by using a technique such as Non-Patent Document 6. By using this operation, a function “parent (x)” that returns the parent node ID to the node x (node ID = “x” node) or a function “that returns the first child ID to the node x” first_child (x) " This is realized by Equation 9.

For example, the parent node of the node 9 is obtained as the node 2 from “parent (9) = select (9 + 1) −9-1 = 12−9-1 = 2”. Further, the first child node of the node 9 is “first_child (9) = select ₀ (9 + 1) −9 = 23−9 = 14”, and is determined to be the node 14.

  However, just because “first_child (x)” returns a node ID of 0 or more, it is not guaranteed that the node x has a child node. The necessary and sufficient condition that the node x has child nodes is “first_child (x) ≠ first_child (x + 1)”. Further, the range of the ID of the child node related to the node x is obtained by [first_child (x), first_child (x + 1)). The n-gram access device 2 accesses the n-gram language model using the above functions.

<Second form>
Next, a second form of the compression unit 4 will be described. Here, the LOUDS used in the first form is expanded so that it can be expressed more compactly using the characteristics of n-gram.

  That is, as shown in FIGS. 2 and 3, in the case of n-gram, there is no information stored in the root node. Therefore, the root node is deleted so that the virtual super root node directly points to each node of 1-gram. Here, the node ID is numbered so that the first node of 1-gram is “0”. This reduces the 2 bits of the old root node.

In the case of n-gram, it has a structure with a hierarchy of 1 to n, and the node of the highest order n in the lowest layer has no child nodes. Here, when the number of X-gram nodes is N _x , the number of nodes of the highest order of n-gram is N _n , and N _n “0” s are redundant. Therefore, the first node ID of the highest order is stored in the main storage device, and when accessing the n-gram language model, it is determined that there is no child node other than the node ID. Thereby, N _n “0” s, that is, N _n bits can be erased.

Furthermore, the super root node, is represented by the formula 16, by storing the number N ₁ of 1-gram, which can be removed. In this way, a tree structure is generated and bits are assigned assuming that the nodes do not exist. This can reduce N ₁ +1 bits. Thus, the compressed pointer in the second form is an expression (extended LOUDS expression) having no bit corresponding to the super node and no bit corresponding to the n-gram of the highest order. The compression-converted pointer is stored in the compressed data storage unit 7.

  In order to access the n-gram language model expressed in a compressed form in the second form, the expression 8. Equation 9 is replaced by Equation 10. Rewrite into Equation 11.

However, assuming that the first node ID of the highest order is Y, when “x ≧ Y”, “select ₁ (x) = select ₁ (Y−1), select ₀ (x) = select ₀ (Y−1) ) + X−Y + 1 ”.

  FIG. 7 shows a structure in which the trie structure is trimmed and optimized for n-gram. Table 2 shows the LOUDS bit string optimized for this structure.

The node 8 in FIG. 7 and Table 2 corresponds to the node 9 in FIG. Since N ₁ = 4 here, the parent node of the node 8 is obtained as the node 1 from “parent (8) = select1 (8 + 1−4) + 4−8 = 5 + 4−8 = 1”. Further, from “first_child (8) = select ₀ (8) + 4 + 1−8 = 16 + 4 + 1−8 = 13”, the first child node of the node 8 is obtained as the node 13.

Here, when the n-gram language model is compressed based on the first form, the pointer expression is expressed by the bit of Expression 15. Since 2 + N _n + N ₁ +1 bits are reduced from here, according to the compression of the second form, the n-gram language model expresses a pointer with the number of bits of Expression 17, and can further suppress the amount of data.

<< n-gram access device 2 >>
The access device 2 refers to the compressed data storage unit 7 using the word string (formula 2) given through the input means as an input. Here, the smoothed probability (Equation 6) and the backoff coefficient (Equation 7) are accessed by tracing the pointer of the n-gram model compressed and expressed by the compression device 1, and these values are output. .

Specifically, the access unit (first_child) 8 is "w _1, ..., w _n" as input the position of the n-gram of "w _1, ..., w _n, w _{n + 1} "(N + 1) -gram", the position of the word ID with the smallest word ID of "w _{n + 1} " is returned to the input means. At this time, if the stored data in the compressed data storage unit 7 is a pointer expression based on the trie structure (compression result of the first form), it is calculated by Expression 9, while a pointer expression based on the n-gram structure (second form) If the result of compression is calculated by the following equation (11).

Moreover, the access unit (parent) 9 is _{"w 1, ..., w n-} 1, w n " as input the position of the n-gram of _{"w 1, ..., w n-} 1 " The position of (n-1) -gram is returned to the input means. At this time, if the stored data in the compressed data storage unit 7 is a pointer expression based on the trie structure (compression result of the first form), it is calculated by Expression 8, while a pointer expression based on the n-gram structure (second form) (Compression result of (1)) is calculated according to Equation 10. It should be noted that Expressions 8 to 11 are defined in a program or the like.

  As described above, according to both the devices 1.2, since the data amount of the pointer array of the n-gram language model is suppressed, the main memory device of a general-purpose computer can be used without using an expensive computer (computer). Most of the pointer array can be stored in the memory, and the compressed data storage unit 7 can be used as a so-called on-memory database.

  Accordingly, the access speed to the pointer of the access unit 8.9 is improved, and the n-gram model can be efficiently accessed during machine translation or speech recognition using the n-gram. In addition, if the probability of the frequently used n-gram language model (Equation 6) and the backoff coefficient (Equation 7) are cached in a storage buffer or the like, the processing can be further speeded up.

≪Experimental example≫
An n-gram compression experiment conducted by the inventors to confirm the effectiveness of the compression apparatus 1 will be described. In this experiment, as shown in Table 3, “English Gigaword 5-gram” learned using “English Gigaword 3rd Edition”, “English Web 1T 5-gram” published by LDC, and GSK “Japan Web 1T 7-gram” published in Japan was used.

  “Count size (gizp)” in Table 3 indicates the size (size) of each n-gram used in the experiment. Here, the size of an ASCII text file storing the frequency of n-gram is compressed with gzip.

  Table 4 shows the compression result of the pointer array in the experiment. “4-byte Pointer” in Table 4 represents a pointer array when the pointer of the “n-gram language model data structure (pointer fixed byte representation)” is represented by a 4-byte integer (the data conversion unit). This corresponds to the pointer array obtained in step 3).

  “Proposed method” indicates a result (a compression result of the second form) in which this pointer array is expressed by a LOUDS bit string optimized to n-gram. According to this "proposed method", as shown in each column of Table 4, a general expression method for expressing a pointer with a 4-byte integer is compressed to about 1/10. As a result, it has been clarified that the data amount of the pointer array is effectively reduced.

≪Programs≫
The present invention can also be configured as a program that causes a computer to function as a part or all of the units 3 to 9 that constitute both the devices 1.2. In this case, the computer is caused to execute all or a part of the data conversion step, the pointer expression compression step, and the access step.

  This program can be provided through a network such as a website or e-mail. The program is recorded on a recording medium such as a CD-ROM, DVD-ROM, CD-R, CD-RW, DVD-R, DVD-RW, MO, HDD, Blu-ray Disk (registered trademark). It is also possible to save and distribute. This recording medium is read using a recording medium driving device, and the program code itself realizes the processing of the above embodiment, so that the recording medium also constitutes the present invention.

DESCRIPTION OF SYMBOLS 1 ... Language model compression apparatus 2 ... Language model access apparatus 3 ... Data structure conversion part (data structure conversion means)
4. Pointer expression compression unit (pointer expression compression means)
DESCRIPTION OF SYMBOLS 5 ... Language model memory | storage part 6 ... Conversion data memory | storage part 7 ... Compressed data memory | storage part 8 ... n-gram access part "first_child" (2nd access means)
9: n-gram access part “parent” (first access means)

Claims (12)

An apparatus for compressing a model representation of an n-gram language model obtained from an appearance frequency of a word string composed of n consecutive words,
A data structure converting means for providing a virtual root node and converting the structure of the n-gram language model into a trie structure;
Compression means for compressing and converting the trie structure converted by the data structure conversion means into LOUDS (Level-Order Undegree Sequence Sequence) expression;
A language model compression apparatus comprising: An apparatus for compressing a model representation of an n-gram language model obtained from an appearance frequency of a word string composed of n consecutive words,
The position (node ID) of the first node of the highest order of the n-gram language model is stored in the storage device, and the pointer expression representing the structure of the n-gram language model is changed to an expression obtained by deleting the highest order n-gram. Data structure conversion means to convert,
A language model compression apparatus comprising: The language model compression apparatus according to claim 2,
The number of 1-grams N ₁ is stored in a storage device, and the structure of the n-gram language model converted by the data structure conversion means is a super node (virtual root node indicating the root node of the tri structure) and the highest Compression means for compressing and converting into a LOUDS expression expanded so as not to have bits corresponding to the n-gram of the order;
A language model compression apparatus, further comprising: An apparatus for accessing an n-gram language model with reference to storage means for storing the pointer compressed and converted by the language model compression apparatus according to claim 1,
First access means for calculating and outputting the position of the parent node “(n−1) -gram” (parent) with respect to the position x of the n-gram of the input word string by Formula 8;

The position of the child node (first_child) having the smallest word ID among the child nodes “(n + 1) -gram” is calculated by Expression 9 with respect to the n-gram position x of the input word string and output. Two access means;

An access device for a language model, comprising: An apparatus for accessing an n-gram language model with reference to storage means for storing the pointer compressed and converted by the language model compression apparatus according to claim 3,
A first access means for calculating and outputting the position of the parent node “(n−1) -gram” (parent) with respect to the position x of the n-gram of the input word string according to Expression 10;

The position of the child node (first_child) having the smallest word ID among the child nodes “(n + 1) -gram” is calculated by Expression 11 and output with respect to the n-gram position x of the input word string. Two access means;

An access device for a language model, comprising: A method for compressing a model representation of an n-gram language model obtained from the frequency of occurrence of a word string consisting of n consecutive words,
A data structure converting means for providing a virtual root node and converting the structure of the n-gram language model into a trie structure;
A compression step in which a compression unit compresses and converts the trie structure converted in the data structure conversion step into a LOUDS (Level-Order Unary Degree Sequence) expression;
A language model compression method comprising: A method for compressing a model representation of an n-gram language model obtained from the frequency of occurrence of a word string consisting of n consecutive words,
The data structure conversion means stores the position (node ID) of the first node of the highest order of the n-gram language model in the storage device, and stores a pointer expression representing the structure of the n-gram language model as the highest order n- A data structure conversion step that converts the gram into a deleted expression,
A language model compression method comprising: The language model compression method according to claim 7, wherein
Compression means, stores the number N ₁ of 1-gram in the storage device, virtual route structure of the converted n-gram language model, which points to the root node of the super node (TRIE structure in the data structure converting means Node) and a compression step that compresses and converts to a LOUDS representation expanded to have no bits corresponding to the highest order n-grams,
A language model compression method further comprising: A method for accessing an n-gram language model by referring to storage means for storing the pointer compressed and converted by the language model compression method according to claim 6,
A step in which the first access means calculates and outputs the position of the parent node “(n−1) -gram” (parent) according to Expression 8 with respect to the n-gram position x of the input word string. When,

The second access means calculates the position of the child node (first_child) having the smallest word ID among the child nodes “(n + 1) -gram” with respect to the position x of the n-gram of the input word string as shown in Expression 9 Calculating and outputting by

A method for accessing a language model, comprising: A method for accessing an n-gram language model with reference to storage means for storing the pointer compressed and converted by the language model compression method according to claim 8,
A step in which the first access means calculates and outputs the position of the parent node “(n−1) -gram” (parent) with respect to the position x of the n-gram of the input word string according to Equation 10; When,

The second access means calculates the position of the child node (first_child) having the smallest word ID among the child nodes “(n + 1) -gram” with respect to the position x of the n-gram of the input word string as shown in Expression 11 Calculating and outputting by

A language model access method characterized by comprising:   The language model compression program for functioning a computer as each means which comprises the language model compression apparatus of any one of Claims 1-3.   A language model access program for causing a computer to function as each means constituting the language model access apparatus according to claim 4.

Images