JP2019012468A - Meaning specification program, information generation program, meaning specification method, information generation method, meaning specification device, and information generation device - Google Patents

Abstract

To provide an encoder that specifies the meanings of a polysemic word (word) included in text data.SOLUTION: An encoder receives text data. The encoder extracts a word included in the received text data. When the extracted word has a plurality of meanings, the encoder refers to, for every meaning of the word having a plurality of meanings, a storage unit that stores co-occurrence information on other words for a word, and specifies the meanings of the extracted word on the basis of the co-occurrence information.SELECTED DRAWING: Figure 9

Description

  The present invention relates to a meaning definition program and the like.

  In recent years, there is a technique for analyzing words (morphemes) constituting text data. There is also a Word2Vec technology that generates vectors from each word. For example, in the Word2Vec technology, a process for calculating a vector of each word is performed based on the relationship between a certain word and another adjacent word. In addition, the words of the text data include a plurality of meanings having a plurality of meanings, and one vector is similarly generated for such meanings using the Word2Vec technique.

  The prior art 1 describes extracting a multiple meaning word assigned with a plurality of meanings in a document such as a proposal or specification regarding an information processing system. Prior art 2 describes that concept vectors are assigned to all words in the word dictionary.

JP2013-020431A JP 2006-338342 A

  However, the above-described conventional technique has a problem that it is not possible to specify the meaning of a multiple meaning word (word) included in text data.

  For example, in the related art, one vector is assigned to each word included in text data, and it does not specify in what sense a polysemy having a plurality of meanings is used.

  In one aspect, the present invention provides a meaning definition program, an information generation program, a meaning definition method, an information generation method, a meaning definition device, and an information generation device that can specify the meaning of a polysemy (word) included in text data. The purpose is to provide.

  In the first plan, the computer executes the following processing. The computer accepts text data. The computer extracts words included in the received text data. When the extracted word includes a plurality of meanings, the computer refers to the storage unit that stores the co-occurrence information of other words for each word meaning including the plurality of meanings, based on the co-occurrence information, Identify the meaning of the extracted word.

  It is possible to specify the meaning of the multiple meaning word (word) included in the text data.

FIG. 1 is a diagram for explaining an example of processing of the encoding device according to the present embodiment. FIG. 2 is a functional block diagram illustrating the configuration of the encoding apparatus according to the present embodiment. FIG. 3 is a diagram illustrating an example of the data structure of the offset table. FIG. 4 is a diagram illustrating an example of a data structure of the meaning HMM. FIG. 5 is a diagram for explaining the pointer of the offset table. FIG. 6 is a diagram illustrating an example of the data structure of the encoded zelkova tree. FIG. 7 is a diagram illustrating an example of the data structure of the dynamic dictionary. FIG. 8 is a diagram for explaining an example of processing of the encoding unit. FIG. 9 is a flowchart illustrating the processing procedure of the encoding apparatus according to the present embodiment. FIG. 10 is a diagram illustrating an example of a hardware configuration of a computer that realizes the same function as that of the encoding device.

  Hereinafter, embodiments of a meaning specification program, an information generation program, a meaning definition method, an information generation method, a meaning definition device, and an information generation device disclosed in the present application will be described in detail based on the drawings. Note that the present invention is not limited to the embodiments.

  FIG. 1 is a diagram for explaining an example of processing of the encoding device according to the present embodiment. The encoding device is an example of a meaning-specific device and an information generation device. The encoding device reads each word from the text file F1 and encodes each word to generate an encoded file F2. When encoding a word, the encoding device specifies the meaning of the word and assigns a code corresponding to the meaning.

  FIG. 1 illustrates a case where the word “deepΔ” is encoded among “..., HeΔhasΔdeepΔideaΔtoΔclearΔthatΔmatter...” Included in the text file F1. . Here, “Δ” indicates a space. The word “deepΔ” is a polysemy having a plurality of meanings. For example, the word `` deep △ '' has the meaning of `` top to bottom, front to back, measurement, deep, breath, sounds, colors, sleep, serious, emotions, knowledge, difficult to understand, involved, person, in sport '' There is.

  Based on the semantic HMM (Hidden Markov Model) 142, the encoding device specifies the meaning of “deepΔ” in the text file F1 from a plurality of meanings. The word meaning HMM 142 associates a multiple meaning word, a word meaning ID, a word meaning, and a plurality of co-occurrence words (co-occurrence rate).

  An ambiguous word indicates a word having a plurality of meanings. The meaning ID is a number that uniquely identifies the meaning included in the polysemy. The meaning means each meaning included in the polysemy. A co-occurrence word indicates a word that co-occurs with a polysemy of a certain meaning. Co-occurrence words are associated with co-occurrence rates. For example, when the ambiguous word “deep” appears in a sentence with the meaning of the word “knowledge”, the possibility that the word “understanding” co-occurs before and after the “deep” is “41%”. It is shown. In addition, when the ambiguous word “deep” appears in the sentence with the meaning of the word “knowledge”, the possibility that the word “idea” co-occurs before and after the “deep” is “19%”. It is shown.

  The encoding device compares the words appearing before and after “deepΔ” in the text file F1 with the co-occurrence words of the meaning HMM 142 to determine a record of meaning that hits. In the example shown in FIG. 1, since “ideaΔ” appears after “deepΔ” in the text file F1, the record with the meaning ID (12) of the meaning HMM 142 is hit. For this reason, the encoding device specifies the meaning of “deepΔ” in the text file F1 as the meaning “knowledge” of the meaning ID (12).

  The encoding device registers the meaning code “A14F06h” corresponding to the meaning “knowledge” of the ambiguous word (word) “deepΔ” in the dynamic dictionary 144. It is assumed that a meaning code corresponding to a set of words and meanings is set in advance.

  The encoding device registers the meaning code “A14F06h” in the dynamic dictionary 144. For example, the dynamic code indicating the registration position of the meaning code “A14F06h” is “AE00h”. The encoding apparatus acquires “AE00h” from the dynamic dictionary 144.

  The encoding device performs encoding by outputting the dynamic code “AE00h” corresponding to the ambiguous word (word) “deepΔ” to the encoded file F2. The encoding device performs encoding by repeatedly executing the above-described processing for other multiple meaning words having a plurality of meanings such as “deepΔ”. On the other hand, the encoding device performs encoding by outputting a code (static code or dynamic code) assigned in advance to the encoded file F2 for a word having a single meaning.

  The encoded file F2 includes a header portion 2a, an encoded data portion 2b, and a trailer portion 2c. The header part 2a includes information on the appearance frequency of words. The encoded data portion 2b stores encoded data generated by the encoding device. The trailer unit 2c includes information on the dynamic dictionary 144 and the like.

  As described above, the encoding device can identify the meaning of a word by comparing the words that appear before and after the target word of the text file F1 with the co-occurrence words of the meaning HMM 142. Also, the encoding device registers the meaning code corresponding to the combination of the target word and the meaning in the dynamic dictionary 144, acquires the dynamic code, and performs encoding. For this reason, even if it is the same word, the encoding by the code | cord | chord according to the meaning can be performed.

  FIG. 2 is a functional block diagram illustrating the configuration of the encoding apparatus according to the present embodiment. As illustrated in FIG. 2, the encoding device 100 includes an interface unit 110, a storage unit 140, and a control unit 150.

  The interface unit 110 is a processing unit that exchanges information with other external devices via a network or the like.

  The storage unit 140 includes a text file F1, an offset table 141, a meaning HMM 142, an encoded zelkova tree 143, a dynamic dictionary 144, a static dictionary 145, and an encoded file F2. The storage unit 140 corresponds to a semiconductor memory element such as a random access memory (RAM), a read only memory (ROM), and a flash memory, and a storage device such as a hard disk drive (HDD).

  The text file F1 is text data to be encoded. The text file F1 has a plurality of words separated by a space “Δ” as described in FIG.

  The offset table 141 is a table that holds a position (offset) of a word included in the text file F1, a pointer indicating whether or not the word is a polysemy, and the like. FIG. 3 is a diagram illustrating an example of the data structure of the offset table. As shown in FIG. 3, the offset table 141 associates numbers, basic codes, offsets, and multi-word pointers.

  The number indicates the registration order of each record in the offset table 141. The basic code is a code assigned in advance to each word and uniquely indicates the word. This basic code is different from the static code and dynamic code described later. The offset indicates the position where the word corresponding to the basic code exists on the text file F1.

  The multiple word pointer indicates whether or not the word corresponding to the basic code is a multiple word having a plurality of word meanings. When the ambiguous word pointer is “0”, it indicates that the word corresponding to the basic code of the corresponding record is not the ambiguous word. For example, since the ambiguity pointer of the word code “A14F04h” corresponding to the number “0” in FIG. 3 is “0”, the word of the word code “A14F04h” is not a ambiguity.

  When the “pointer” to the meaning HMM 142 is stored in the ambiguous word pointer, it indicates that the word corresponding to the corresponding basic code is an ambiguous word. The pointer indicates the record position of the meaning HMM 142, and the record position has information on each meaning of the word that becomes a polysemy. For example, since the pointer to the meaning HMM 142 is stored in the word code “A14F06h” corresponding to the number “2” in FIG. 3, the word corresponding to the word code “A14F06h” is an ambiguous word.

  The meaning HMM 142 is a table that holds information about other words that co-occur before and after a word used in a certain meaning for each meaning included in the multiple meaning. FIG. 4 is a diagram illustrating an example of a data structure of the meaning HMM. As shown in FIG. 4, this meaning HMM 142 associates multiple meanings, meaning IDs, meanings, and a plurality of co-occurrence words (co-occurrence rates). The explanation regarding the multiple meaning word, the meaning ID, the meaning, and the plurality of co-occurrence words (co-occurrence rate) is the same as that explained in FIG.

  FIG. 5 is a diagram for explaining the pointer of the offset table. For example, the pointer corresponding to the number “2” in the offset table 141 indicates the record position of the multiple meaning “deep” of the meaning HMM 142. For this reason, it is understood that the meaning of the meaning IDs (1) to (16) is included in the ambiguous word “deep” corresponding to the word code “A14F06h”.

  The encoding zelkova tree 143 has information on static codes and dynamic codes used when encoding words. FIG. 6 is a diagram illustrating an example of the data structure of the encoded zelkova tree. As shown in FIG. 6, this encoding zelkova tree 143 has 2 grams, a bitmap, a pointer, an English word, a static code, a dynamic code, a basic code, and a semantic code.

  2 grams is information indicating a two-character string (word). The bitmap indicates a bitmap corresponding to a character string of 2 grams. For example, the bitmap corresponding to “aa” is “0 — 0 — 0 — 0 — 0”. The pointer is a pointer indicating the position of the English word corresponding to the bitmap.

  For example, when a static code corresponding to the word “ableΔ” is searched from the encoded zelkova tree 143, bitmaps corresponding to the two grams “ab”, “bl”, “le”, “eΔ” are selected. Combine each. When the values of all the bitmaps are 0 in each digit of the bitmap, the corresponding digit of the combined bitmap is set to “0”. On the other hand, if at least one “1” is included, the corresponding digit is set to “1” to combine the bitmaps.

  For example, the bitmap of “ab” is “1 — 0 — 0 — 0 — 0”, the bitmap of “bl” is “0 — 1 — 0 — 0 — 0”, the bitmap of “le” is “0 — 0 — 1 — 0 — 0”, and the bitmap of “eΔ” is “0 — 0 — 0 — 1 — 0”. In this case, the bitmap obtained by combining the bitmaps is “1_1_1_1_0”. A static code corresponding to the word “ableΔ” is stored at the position indicated by the pointer in the bitmap “1_1_1_1_0”.

  English words are words appearing in, for example, an English dictionary. A static code is a code assigned to an English word. One static code is assigned to one English word. For this reason, even if it is a polysemy, the static code assigned to this polysemy is one.

  When the English word is a multiple meaning word, the static code record stores a meaning ID that uniquely indicates the meaning of the multiple meaning word.

  A dynamic code is a code that is dynamically assigned to each meaning of a polysemy. The process of assigning a dynamic code to the meaning of a multiple meaning will be described later.

  The basic code is a code assigned in advance to each English word (word) and uniquely indicates the word. The meaning code is a code assigned in advance to the meaning of the word.

  The dynamic dictionary 144 is information used when a dynamic code is assigned to a word meaning. FIG. 7 is a diagram illustrating an example of the data structure of the dynamic dictionary. The dynamic dictionary 144 includes a dynamic dictionary unit 144a and a buffer unit 144b. The dynamic dictionary unit 144a includes a dynamic code, a meaning code, polysemy information, and a pointer.

  The dynamic code indicates the registration position of the dynamic dictionary 144. The dynamic code is a code assigned to the word stored in the buffer 144b, and even if the word is the same, the assigned dynamic code is different if the meaning is different.

  The meaning code indicates the meaning code of the word to which the dynamic code is assigned. Polysemy information indicates the meaning and meaning ID of a word to which a dynamic code is assigned. The pointer is a pointer that indicates the storage position of the multi-word buffer 144b corresponding to the dynamic code.

  For example, the dynamic dictionary 144 shown in FIG. 7 indicates that the dynamic code “AE00h” is assigned to the polysemy “deep” having the meaning of “knowledge”. The pointer of the dynamic code “AE00h” indicates that the ambiguous word “deepΔ” is stored at the head position of the buffer 144b. The pointer of the dynamic code “AE01h” indicates the position next to “deepΔ” in the buffer 144b.

  The static dictionary 145 is information that associates words with static codes.

  The encoded file F2 is a file obtained by encoding the text file F1. The description of the encoded file F2 is the same as that of the encoded file F2 described in FIG.

  Returning to the description of FIG. The control unit 150 includes an extraction unit 151, a specification unit 152, an encoding unit 153, and a co-occurrence information generation unit 154. The control unit 150 can be realized by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like. The control unit 150 can also be realized by a hard wired logic such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

  The extraction unit 151 is a processing unit that extracts data of a content part to be encoded from the text file F1 in units of words. For example, when the data of the content part is “He △ has △ deep △ idea △ to △ clear △ that △ matter.”, “He △”, “has △”, “deep △”, “△ idea ”,“ ToΔ ”,“ clearΔ ”,“ thatΔ ”,“ matter. ”, Each word is extracted. The extraction unit 151 outputs each extracted word to the specifying unit 152.

  The specifying unit 152 is a processing unit that acquires a word and specifies the meaning of the word based on the offset table 141, the meaning HMM 142, and the encoded zelkova tree 143. Hereinafter, an example of processing of the specifying unit 152 will be described.

  The specifying unit 152 specifies the basic code of the word. For example, the specifying unit 152 compares the word and the encoded zelkova tree 143 to specify the basic code of the word.

  The specifying unit 152 compares the basic code of the word with the offset table 141 and determines whether or not the word is an ambiguous word. For example, when the pointer corresponding to the basic code is “0”, the specifying unit 152 determines that the word is not a polysemy. On the other hand, when the pointer corresponding to the basic code indicates the record position of the semantic HMM 142, the specifying unit 152 determines that the word is an ambiguous word.

  If the specifying unit 152 determines that the word is not a multiple meaning word, the specifying unit 152 outputs the word and information indicating that the word is not a multiple meaning word to the encoding unit 153.

  If the identifying unit 152 determines that the word is an ambiguous word, the identifying unit 152 performs the following process. The specifying unit 152 refers to the pointer corresponding to the basic code of the word in the offset table 141 and specifies the record of the meaning HMM 142 indicated by the pointer.

  The specifying unit 152 refers to the record specified by the meaning HMM 142 and specifies the meaning of the word. For example, the specifying unit 152 compares the words appearing before and after the word that is the target of meaning definition with the co-occurrence words (co-occurrence rate) of the meaning HMM 142 to specify the meaning of the word. The co-occurrence word to be compared is a co-occurrence word having a meaning associated with the corresponding multiple meaning word. For example, if the ambiguous word is “deep”, it is compared with the co-occurrence words corresponding to the meanings (1) to (16) of the ambiguous word “deep”.

  Here, as an example, a case will be described in which the specifying unit 152 specifies the meaning of the ambiguous word “deep Δ” included in the text file F1 “He Δhas Δdeep Δidea Δto Δclear Δthat Δmatter. Since “ideaΔ” appears after “deepΔ” in the text file F1, the specifying unit 152 hits the record of the meaning ID (12) of the meaning HMM 142 shown in FIG. For this reason, the encoding device specifies the meaning of “deepΔ” in the text file F1 as the meaning “knowledge” of the meaning ID (12).

  When the co-occurrence word of each meaning is hit, the specifying unit 152 may determine the meaning according to the number of hit co-occurrence words, the co-occurrence rate, and the like. For example, the specifying unit 152 hits one co-occurrence word among the co-occurrence words corresponding to the meaning ID (1), and determines two co-occurrence words among the co-occurrence words corresponding to the meaning ID (12). In the case of a hit, the meaning of the meaning ID (12) is specified as the meaning of the word. Alternatively, the specifying unit 152 hits a co-occurrence word having a co-occurrence rate of 12% among the co-occurrence words corresponding to the meaning ID (1), and out of the co-occurrence words corresponding to the meaning ID (12). When a co-occurrence word with an occurrence rate of 10% is hit, the meaning of the meaning ID (1) is specified as the meaning of the word. The administrator may appropriately adjust whether to give priority to the number of hits to the co-occurrence word or to give priority to the co-occurrence rate.

  If the identifying unit 152 determines that the word is an ambiguous word, the identifying unit 152 outputs the word, information indicating that the word is an ambiguous word, and the meaning ID of the word to the encoding unit 153.

  The encoding unit 153 is a processing unit that performs encoding by assigning a code to a word and outputting the code assigned to the word to the encoded file F2.

  The encoding unit 153 acquires a word to be encoded, and describes a process when the acquired word is “not a polysemy”. The encoding unit 153 compares the word and the encoded zelkova tree 143, and specifies a code (static code or dynamic code) corresponding to the word. The encoding unit 153 outputs the specified code to the encoded file F2. For example, when the word hits the static dictionary 145, the encoding unit 153 identifies the static code of the word. When the word does not hit the static dictionary 145, the encoding unit 153 uses the dynamic dictionary 144 to identify the dynamic code. For example, if the word is not registered in the dynamic dictionary 144, the encoding unit 153 registers the word in the dynamic dictionary 144 and obtains a dynamic code corresponding to the registered position. If the word has been registered in the dynamic dictionary 144, the encoding unit 153 obtains a dynamic code corresponding to the registered position that has already been registered.

  The encoding unit 153 acquires a word to be encoded, and describes a process in a case where the acquired word is “polysemy”. When the word is “polysemy”, the meaning ID is notified from the specifying unit 152 as described above. The encoding unit 153 compares the word and the encoded zelkova tree 143 to identify a dynamic code corresponding to the word.

  As described with reference to FIG. 6, when a word is an ambiguous word, a meaning ID is set in a static code corresponding to the word. Therefore, the encoding unit 153 compares the meaning ID notified from the specifying unit 152 with the meaning ID of the static code, and whether or not the dynamic code corresponding to the meaning ID of the polysemy has already been registered. Determine whether.

  When the dynamic code corresponding to the meaning ID of the polysemy has already been registered, the encoding unit 153 outputs the dynamic code corresponding to the meaning ID to the encoded file F2 as the word code. For example, in the encoded zelkova tree 143 shown in FIG. 6, the dynamic code “AE00h” is registered as the dynamic code with the multiple meaning “deep” and the meaning ID “(12)”. Therefore, the encoding unit 153 outputs the dynamic code “AE00h” to the encoded file F2 as a code corresponding to the meaning “(12)” of the ambiguous word “deep”.

  When the dynamic code corresponding to the meaning ID of the polysemy is not registered, the encoding unit 153 executes the processing described in FIG. FIG. 8 is a diagram for explaining an example of processing of the encoding unit. In the example illustrated in FIG. 8, it is assumed that the dynamic code corresponding to the multiple meaning “deep” and the meaning ID “(12)” of the encoded zelkova tree 143 is not registered.

  The encoding unit 153 corresponds to the smallest dynamic code among the unstored records of the dynamic dictionary unit 144a with the meaning code “A14F06h” corresponding to the polysemy “deep” and the meaning ID “(12)”. Register in the meaning code. In the example illustrated in FIG. 8, the encoding unit 153 registers the meaning code “A14F06h” in the meaning code corresponding to the dynamic code “AE00h” of the dynamic dictionary unit 144a. The encoding unit 153 stores the meaning “knowledge (12)” in the polysemy information corresponding to the meaning code “A14F06h”. It is assumed that the encoding unit 153 acquires the polynomial information from the specifying unit 152.

  The encoding unit 153 registers the ambiguous word “deepΔ” at the position indicated by the pointer corresponding to the dynamic code “AE00h” on the buffer unit 144b. The encoding unit 153 sets the position of the pointer corresponding to the dynamic code “AE01h” immediately after “deepΔ” on the buffer unit 144b.

  By performing the above processing, the encoding unit 153 associates the meaning code “A14F06h” corresponding to the ambiguous word “deep” and the meaning ID “(12)” with the dynamic code “AE00h”. Become. The encoding unit 153 registers the dynamic code “AE00h” in the dynamic code corresponding to the meaning ID “(12)” of the polysemy “deep”. As a result, the dynamic code “AE00h” is assigned to the polysemy “deep” as the meaning of the meaning ID “12”. The encoding unit 153 outputs the dynamic code corresponding to the meaning ID to the encoded file F2 as a word code.

  The encoding unit 153 generates the encoded file F2 by repeatedly executing the above process for each word acquired from the specifying unit 152.

  Returning to the description of FIG. The co-occurrence information generation unit 154 obtains information on an English-English dictionary (not shown) from an external device via the interface 110, and generates a semantic HMM 142 based on the obtained English-English dictionary information. Part. The co-occurrence information generation unit 154 performs processing prior to the processing of the extraction unit 151, the identification unit 152, and the encoding unit 153, and generates the semantic HMM 142. The co-occurrence information generation unit 154 also has a function of a reception unit that receives information such as an English-English dictionary.

  For example, the English-English dictionary information includes information that associates words, meanings included in the words, and explanations of meanings. The co-occurrence information generation unit 154 scans information in the English-English dictionary, associates multiple meanings and meanings, and assigns a unique meaning ID to each meaning. In addition, the co-occurrence information generation unit 154 refers to the information in the English-English dictionary and calculates the co-occurrence rate of the co-occurrence words co-occurring with the word for each word meaning, so that the co-occurrence words corresponding to each word meaning And calculate the co-occurrence rate. Note that the co-occurrence information generation unit 154 acquires information on a sentence in which use of a word is distinguished for each meaning, and calculates the co-occurrence word and the co-occurrence rate corresponding to each meaning using the information on the sentence. May be. The co-occurrence information generation unit 154 generates the meaning HMM 142 by executing the above process.

  Next, an example of a processing procedure of the encoding device 100 according to the present embodiment will be described. FIG. 9 is a flowchart illustrating the processing procedure of the encoding apparatus according to the present embodiment. As illustrated in FIG. 9, the extraction unit 151 of the encoding device 100 extracts the text data F1 to be encoded in units of words (step S101).

  The specifying unit 152 of the encoding apparatus 100 determines whether or not the word is a polysemy based on the offset table 141 (step S102). If the word is not an ambiguous word (step S102, No), the process proceeds to step S103. If the word is an ambiguous word (step S102, Yes), the process proceeds to step S105.

  The encoding unit 153 of the encoding device 100 compares the word and the encoded zelkova tree 143 to identify a code (static code or dynamic code) corresponding to the word (step S103). The encoding unit 153 writes the code (static code or dynamic code) in the encoded file F2 (step S104), and proceeds to step S109.

  The encoding unit 153 identifies the meaning ID based on the words included before and after the target word and the co-occurrence words (co-occurrence rate) of the meaning HMM 142 (step S105). The encoding unit 153 determines whether or not the dynamic code corresponding to the ambiguous word and the meaning ID is registered in the encoded zelkova tree 143 (step S106).

  If the dynamic code corresponding to the ambiguous word and the meaning ID is registered in the encoded zelkova tree 143 (step S106, Yes), the encoding unit 153 proceeds to step S108. On the other hand, when the dynamic code corresponding to the ambiguous word and the meaning ID is not registered in the encoded zelkova tree 143 (No at Step S106), the encoding unit 153 proceeds to Step S107.

  The encoding unit 153 assigns a dynamic code corresponding to the ambiguous word and the meaning ID, and updates the dynamic dictionary 144 (step S107). The encoding unit 153 writes the dynamic code in the encoded file F2 (step S108).

  If it is not the end point of the text data (No at Step S109), the encoding apparatus 100 proceeds to Step S101. When the end point of the text data is reached (Yes at Step S109), the encoding apparatus 100 proceeds to Step S110.

  The encoding unit 153 stores the information of the dynamic dictionary 144 in the trailer unit 2c of the encoded file F2 (Step S110).

  Next, effects of the encoding device 100 according to the present embodiment will be described. The encoding unit 100 can identify the meaning by comparing the words appearing before and after the target word of the text file F1 with the co-occurrence words of the meaning HMM 142. Also, the encoding device registers the meaning code corresponding to the combination of the target word and the meaning in the dynamic dictionary 144, acquires the dynamic code, and performs encoding. For this reason, even if it is the same word, the encoding by the code | cord | chord according to the meaning can be performed.

  The encoding apparatus 100 generates word co-occurrence information for each word meaning based on information such as an English-English dictionary, and generates a word meaning HMM 142 by associating the word meaning with the co-occurrence information. By using the meaning HMM 142, it is possible to specify the meaning of the words included in the text file F1.

  Here, an example of a hardware configuration of a computer that realizes the same function as that of the encoding device 100 described in the present embodiment will be described. FIG. 10 is a diagram illustrating an example of a hardware configuration of a computer that realizes the same function as that of the encoding device.

  As illustrated in FIG. 10, the computer 200 includes a CPU 201 that executes various arithmetic processes, an input device 202 that receives data input from a user, and a display 203. The computer 200 also includes a reading device 204 that reads a program and the like from a storage medium, and an interface device 205 that exchanges data with another computer via a wireless network. The computer 200 also includes a RAM 206 that temporarily stores various information and a hard disk device 207. The devices 201 to 207 are connected to the bus 208.

  The hard disk device 207 includes an extraction program 207a, a specific program 207b, an encoding program 207c, and a co-occurrence information generation program 207d. The CPU 201 reads out the extraction program 207 a, the identification program 207 b, the encoding program 207 c, and the co-occurrence information generation program 207 d and develops them in the RAM 206.

  The extraction program 207a functions as the extraction process 206a. The specific program 207b functions as the specific process 206b. The encoding program 207c functions as the encoding process 206c. The co-occurrence information generation program 207d functions as a co-occurrence information generation process 206d.

  The processing of the extraction process 206a corresponds to the processing of the extraction unit 151. The process of the specifying process 206b corresponds to the process of the specifying unit 152. The process of the encoding process 206c corresponds to the process of the encoding unit 153. The process of the co-occurrence information generation process 206d corresponds to the process of the co-occurrence information generation unit 154.

  Note that the programs 207a to 207d are not necessarily stored in the hard disk device 207 from the beginning. For example, each program is stored in a “portable physical medium” such as a flexible disk (FD), a CD-ROM, a DVD disk, a magneto-optical disk, and an IC card inserted into the computer 200. Then, the computer 200 may read and execute each of the programs 207a to 207d.

  The following supplementary notes are further disclosed with respect to the embodiments including the above examples.

(Appendix 1) Accept text data,
Extract words included in the received text data,
When the extracted word includes a plurality of meanings, for each meaning of the word including a plurality of meanings, referring to a storage unit that stores co-occurrence information of other words for the word, based on the co-occurrence information, A meaning specifying program that causes a computer to execute a process for specifying the meaning of the extracted word.

(Supplementary Note 2) The co-occurrence information includes information on a meaning code corresponding to each of a plurality of meanings of a word, and based on the co-occurrence information and the meanings specified by the process of specifying the meanings, The semantic identification program according to appendix 1, further causing a computer to execute processing for determining a semantic code to be assigned to a word.

(Supplementary note 3) The supplementary note 2 further causes the computer to execute a process of registering the semantic code determined by the process of determining the semantic code at a registration position of a dynamic dictionary to which a registration number is assigned. Meaning definition program.

(Supplementary Note 4) The supplementary note 3 further causes the computer to execute a process of outputting a registration number of a registration position of the dynamic dictionary as a code corresponding to the word extracted by the process of extracting the word. Meaning definition program.

(Supplementary Note 5) Accept information including a word including a plurality of meanings and an explanation of each meaning of the word,
Based on the received information, for each word meaning included in the information, generate co-occurrence information of other words for the meaning,
Generating information that associates the generated co-occurrence information with the meaning of the word included in the information;
An information generation program that causes a computer to execute processing.

(Supplementary Note 6) A semantic identification method executed by a computer,
Accept text data,
Extract words included in the received text data,
When the extracted word includes a plurality of meanings, for each meaning of the word including a plurality of meanings, referring to a storage unit that stores co-occurrence information of other words for the word, based on the co-occurrence information, A meaning specifying method, comprising: performing a process for specifying a meaning of the extracted word.

(Supplementary Note 7) The co-occurrence information includes information on meaning codes corresponding to a plurality of meanings of a word, and based on the co-occurrence information and the meanings specified by the process of specifying the meanings, The semantic identification method according to appendix 6, further comprising the step of determining a semantic code to be assigned to the word.

(Additional remark 8) The meaning as described in additional remark 7 which further performs the process which registers the meaning code determined by the process which determines the said meaning code in the registration position of the dynamic dictionary to which the registration number was allocated. Identification method.

(Additional remark 9) The meaning of additional remark 8 characterized by further performing the process which outputs the registration number of the registration position of the said dynamic dictionary as a code corresponding to the word extracted by the process which extracts the said word Identification method.

(Supplementary Note 10) An information generation method executed by a computer,
Receiving information including a word including a plurality of meanings and an explanation of each meaning of the word;
Based on the received information, for each word meaning included in the information, generate co-occurrence information of other words for the meaning,
Generating information that associates the generated co-occurrence information with the meaning of the word included in the information;
An information generation method characterized by executing processing.

(Additional remark 11) The extraction part which receives text data and extracts the word contained in the received said text data,
When the word includes a plurality of meanings, each word meaning including a plurality of meanings is extracted based on the co-occurrence information with reference to a storage unit that stores co-occurrence information of other words for the word And a specifying unit that specifies the meaning of the word.

(Supplementary Note 12) The co-occurrence information includes information on meaning codes corresponding to a plurality of meanings of a word, and the specifying unit includes the meaning specified by the co-occurrence information and a process for specifying the meaning. The semantic identification device according to appendix 11, further comprising a process of determining a semantic code to be assigned to a word based on the above.

(Supplementary note 13) The supplementary note 12, further comprising an encoding unit for registering the semantic code determined by the processing for determining the semantic code at a registration position of a dynamic dictionary to which a registration number is assigned. Semantic identification device.

(Supplementary note 14) The supplementary note 13, wherein the encoding unit further executes a process of outputting a registration number of a registration position of the dynamic dictionary as a code corresponding to the word extracted by the extraction unit. Description meaning device.

(Supplementary Note 15) A reception unit that receives information including a word including a plurality of meanings and an explanation of each meaning of the word;
Based on the information received by the receiving unit, for each word meaning of the word included in the information, co-occurrence information of another word corresponding to the word meaning is generated, and the generated co-occurrence information for the word meaning of the word included in the information is generated. A co-occurrence information generation unit that generates information for associating the occurrence information.

DESCRIPTION OF SYMBOLS 100 Coding apparatus 110 Interface part 140 Storage part 141 Offset table 142 Word meaning HMM
143 Coding zelkova 144 Dynamic dictionary 150 Control unit 151 Extraction unit 152 Identification unit 153 Coding unit 154 Co-occurrence information generation unit

Claims (9)

Accept text data,
Extract words included in the received text data,
When the extracted word includes a plurality of meanings, for each meaning of the word including a plurality of meanings, referring to a storage unit that stores co-occurrence information of other words for the word, based on the co-occurrence information, A meaning specifying program that causes a computer to execute a process for specifying the meaning of the extracted word.   The co-occurrence information includes information on the meaning code corresponding to each of a plurality of meanings of the word, and the meaning assigned to the word based on the co-occurrence information and the meaning specified by the processing for specifying the meaning The semantic identification program according to claim 1, further causing a computer to execute a process of determining a code.   The meaning specification according to claim 2, further causing the computer to execute a process of registering the meaning code determined by the process of determining the meaning code at a registration position of a dynamic dictionary to which a registration number is assigned. program.   4. The word meaning specification according to claim 3, further causing the computer to execute a process of outputting a registration number of a registration position of the dynamic dictionary as a code corresponding to the word extracted by the process of extracting the word. program. Receiving information including a word including a plurality of meanings and an explanation of each meaning of the word;
Based on the received information, for each word meaning included in the information, generate co-occurrence information of other words for the meaning,
Generating information that associates the generated co-occurrence information with the meaning of the word included in the information;
An information generation program that causes a computer to execute processing. A method of identifying meanings executed by a computer,
Accept text data,
Extract words included in the received text data,
When the extracted word includes a plurality of meanings, for each meaning of the word including a plurality of meanings, referring to a storage unit that stores co-occurrence information of other words for the word, based on the co-occurrence information, A meaning specifying method, comprising: performing a process for specifying a meaning of the extracted word. An information generation method executed by a computer,
Receiving information including a word including a plurality of meanings and an explanation of each meaning of the word;
Based on the received information, for each word meaning included in the information, generate co-occurrence information of other words for the meaning,
Generating information that associates the generated co-occurrence information with the meaning of the word included in the information;
An information generation method characterized by executing processing. An extractor that accepts text data and extracts words included in the accepted text data;
When the word includes a plurality of meanings, each word meaning including a plurality of meanings is extracted based on the co-occurrence information with reference to a storage unit that stores co-occurrence information of other words for the word And a specifying unit that specifies the meaning of the word. A reception unit that receives information including a word including a plurality of meanings and an explanation of each meaning of the word;
Based on the information received by the receiving unit, for each word meaning of the word included in the information, co-occurrence information of another word corresponding to the word meaning is generated, and the generated co-occurrence information for the word meaning of the word included in the information is generated. A co-occurrence information generation unit that generates information for associating the occurrence information.

Images