JP2013257756A - Analyzing method of character information, information analyzing device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a system achieving highly accurate morphological analysis which reduces manual maintenance work of a morphological analysis dictionary by paying attention to a meaning which a word (vocabulary) has.SOLUTION: An information analyzing device includes: a first means for reading dictionary data which comprises a word dictionary in which a word used in a natural language and its characteristic are associated with each other and registered, and a concept dictionary in which a semantic tag showing the meaning of a word and a semantic concept showing a concept of the semantic tag are associated with each other for each meaning of the word and registered; a second means for estimating the characteristic of the semantic concept, on the basis of the dictionary data, by a characteristic collection of the words included in the semantic concept of the text data; a third means for reading the concept dictionary and estimating a characteristic content of the word by associating it with the characteristic content of the semantic concept acquired by the second means; and a fourth means for, on the basis of the dictionary data acquired by the first means and the characteristic of the word acquired by the third means, analyzing into an appropriate form for executing information analysis and constructing an analysis dictionary.

Description

  The present invention relates to an information analysis method, an information analysis apparatus, and a program using a computer. In particular, the present invention is directed to text data composed of a natural language included in an electronic document such as content existing in the Web, a company, or locally. The present invention relates to an information analysis method and apparatus for analyzing the word composition of text data and a program used therefor.

In recent years, with the development of the Internet and the spread of computers, electronic documents written in natural language have been distributed in large quantities, and it is expected that they will increase at an accelerated rate in the future.
Under such circumstances, there is an increasing demand for a device for analyzing these digitized documents on a computer and utilizing the digitized documents such as information retrieval, automatic classification, and automatic summarization.

  In order to realize such a utilization mechanism for digitized documents with realistic performance, it is necessary to decompose text data existing in the digitized document into character strings in some form. In particular, there is no word delimiter (such as a space in English) in Japanese, Chinese, Korean, etc., so it is necessary to consider a character string decomposition method.

Generally, as a technique for realizing character string decomposition, a morphological analysis technique (for example, Patent Document 1, and Non-Patent Documents 1, 2, 5, and 6) or an N-gram technique (for example, Patent Document 2 and Non-Patent Document). Documents 3 and 4) are often used. Note that the N-gram technique is a technique of decomposing into a specific character string based on the appearance tendency of a statistically continuous character string, so there is an advantage that a dictionary essential in the morphological analysis technique is not necessary. There is no guarantee that the resulting split string is a linguistically meaningful chunk. For this reason, when character string division based on the meaning of a word or the like is desired, it is mainstream to use a morphological analysis technique.
Each character string after character string division (hereinafter referred to as segmentation) using morphological analysis technology is a linguistically meaningful chunk called a morpheme, and further detailed characteristic information (part of speech) in each morpheme , Nouns, verbs, etc.) can be used, which has the advantage of being able to be reused in more detailed data. However, a program that implements morphological analysis technology (hereinafter referred to as a morphological analysis engine) generally includes a dictionary for executing morphological analysis, in addition to a program serving as a main body for executing morphological analysis. Requires a word dictionary that defines word information corresponding to the language, a connection dictionary that stores information related to vocabulary connection relationships in the language, and the like. That is, in order to perform morphological analysis, a plurality of morphological analysis dictionaries (hereinafter referred to as morphological analysis dictionaries) that store language knowledge corresponding to the language are separately required.
For this reason, many forms of morpheme analysis engines separately provide a morpheme analysis dictionary for each natural language corresponding to the morpheme analysis engine. By using this morphological analysis dictionary, it is said that morphological analysis accuracy of about 95 to 98% can be obtained.

JP-A-8-335222. JP-A-11-338883. JP2009-181408A.

Mao Nagao, “Research on the analysis of Japanese texts by computer”, Ministry of Education Scientific Research Fund Specific Research (1) Research Report, 1983. Akio Yasuda, “Morphological analysis and handwriting”, http://wordminer.comquest.co.jp/wmtips/pdf/H15_01-4.pdf. Takenobu Tokunaga: “Information Retrieval and Language Processing”, University of Tokyo Press, 1999. Kenji Kita: “Probabilistic Language Model”, University of Tokyo Press, 1999. Manabu Okumura: “Basics of Natural Language Processing”, Corona, 2010. Taku Kudo: “Japanese morphological analysis using Conditional Random Fields”, IPSJ, 2004-NL-161, 2004. Tadashi Yanagihara, Kazufumi Ikeda, Kazunori Matsumoto, Yasuhiro Takishima: “Automatic Acquisition Method for Morphological Analysis Dictionaries Based on Information Criterion”, FIT2009, 8th Information Science and Technology Forum, E-027, 2009. Pronceton WordNet: http://wordnet.princeton.edu/. Japanese WordNet: http://nlpwww.nict.go.jp/wn-ja/. Japan Electronic Dictionary Institute: EDR Electronic Dictionary Usage Manual, 1995.

  However, the currently used morphological analysis dictionary is constructed based on past newspaper article data. Therefore, it is suitable for analyzing general text data such as newspapers, but it is constructed from past newspaper article data, so new words, coined words, emoticons, or technical terms Such a vocabulary must be unsupported. Therefore, there is a problem that the analysis accuracy decreases when morphological analysis is performed on text data including words such as new words, coined words, emoticons, or technical terms.

For example, there are unique terms used only within a specific industry or company, such as “WSDL” and “department name”, but it is difficult to define all such terms in the first place.
Moreover, even if all terms in the field are defined, if the morphological analysis dictionary is applied to another industry or company, it is a term that is not used in that industry or company, so the analysis accuracy will be lowered. become. For this reason, for technical terms and the like, a mechanism is required in which a plurality of dictionaries are prepared according to the situation and the dictionaries can be easily switched.

In addition, words are words that define new vocabulary (new words) or coined words that have not existed before, such as “smartphones” and “garakae”, and the meanings of the vocabulary change. It is. For these, a mechanism that can be easily added or changed is required.
In order to solve the above problem, it is necessary to maintain the morphological analysis dictionary according to the situation. However, in the current morphological analysis engine, it is necessary to perform this maintenance manually, which is an excessive burden.

This problem will be described with reference to the drawings.
First, the morphological analysis dictionary includes at least a “word dictionary” for storing information of the word itself and a “connection dictionary” for storing connection information between words.
FIG. 3 is a diagram illustrating an example of a word dictionary that is a part of the morphological analysis dictionary.

The word dictionary exemplified here stores information of words in natural language, and data such as “surface layer”, “left context ID”, “right context ID”, “occurrence cost” as shown in FIG. It is constituted by.
The “surface form” is a form in which words are written.
"Left context ID" is an identifier in the internal state when the word is viewed from the left (front),
“Right context ID” is an identifier in the internal state when the word is viewed from the right (back).
The “occurrence cost” is an index indicating the ease of appearance of the word. The smaller this value, the easier it will appear.
In addition to the data items described above, there are also dictionaries to which parts of speech, basic forms, utilization forms, original forms of words, readings, and the like are assigned.

  Next, FIG. 4 is a diagram illustrating an example of a concatenation dictionary that is a part of the morphological analysis dictionary. The concatenation dictionary exemplified here stores concatenation costs (ease of connection) between words in a natural language, and as shown in FIG. 4, "right context ID" and "left context ID" are paired. In this case, the “joint cost” is configured in a matrix. It means that the smaller the “concatenation cost”, the easier it is to connect between the words holding the context ID (words holding the right context ID and words holding the left context ID). The “right context ID” and the “left context ID” shown here indicate the same as those in FIG.

  In the morphological analysis engine, an internal state called a lattice structure as shown in FIG. 36 is generated using the word dictionary and the concatenated dictionary, and division is executed based on the internal state. Note that the lattice structure generation method and the detailed processing method in the morphological analysis technique are well-known and will not be described.

  FIG. 36 is a diagram illustrating an example in which the character string “living in Tokyo” has a lattice structure. Here, for example, the numerical value described under the word “Tokyo” in this figure is called “occurrence cost”, and in this case, the value is “15”. Further, for example, a numerical value existing between “Tokyo” and “Tokyo” is called “joint cost”, and is “10” here. In the morphological analysis technology, the “occurrence cost” and the “joint cost” existing on the path in the lattice structure are added, and the path having the lowest sum (in this case, “Tokyo / city / in / living” ")" Is determined to be the most suitable division, and is presented as a result of the split writing execution.

  On the basis of this mechanism, if the accuracy of numerical values corresponding to “occurrence cost” and “joint cost” set in the dictionary is low, this will directly lead to a decrease in morphological analysis accuracy. In other words, when it is desired to newly register a word that does not exist in the dictionary, after calculating appropriate numerical values of “occurrence cost” and “joint cost” corresponding to the word, the corresponding word is registered in the morphological analysis dictionary. This work is essential. However, since this is a newly registered word, there is a problem that there is no value that serves as an index for setting a necessary numerical value. For this reason, in practice, dictionary maintenance workers often use empirically calculated values that are considered reasonable while performing trial and error, and register words.

  In the conventional maintenance work of the morphological analysis dictionary, since the above-described problems are solved manually by trial and error, it has been an excessive burden. In other words, since the work cost for maintenance of the morphological analysis dictionary is quite high in the first place, there is no mechanism for easily adding words to the morphological analysis dictionary or switching the morphological analysis dictionary according to the usage.

  As a method for reducing the work cost, for example, there is a method presented in Non-Patent Document 7 described above. This method is a method for estimating a word boundary and its part of speech in an unknown word by a model test based on an information criterion for a vocabulary detected as an unknown word. However, this method is only a method for statistically estimating word boundaries in unknown words, that is, a method for estimating whether word break positions exist in unknown words. There is a problem that it is impossible to deal with a case where a vocabulary that should not be divided by a morphological analysis engine.

  For example, consider a case where morphological analysis is performed on text data including “Hitachi Solutions” in a state where the word “Hitachi Solutions” is not registered in the morphological analysis dictionary. In this case, words such as “Hitachi” and “Solutions” are extracted as an analysis result by the morphological analysis engine. However, "Hitachi Solutions" is a company name and should be extracted as a single word. In the method of Non-Patent Document 7, for example, it is possible to estimate whether or not there is a word break position in “Solutions”, but in the first place, it is divided into “Hitachi” and “Solutions” by the morphological analysis engine. It is difficult to cope with this method.

  Therefore, the present invention provides a method, system, and program for realizing high-precision morphological analysis while reducing manual maintenance work of the morphological analysis dictionary by focusing on the meaning of words (vocabulary). Objective.

In order to solve the above problems, the information analysis method according to the present invention focuses on the meaning of words (vocabulary), and even if the words “occurrence cost” and “joint cost” do not exist, the words (vocabulary) Means for identifying the cost from the meaning of.
The present invention is not limited to Japanese, and is inspired by the fact that, as a characteristic of a language, text belonging to the same type (concept) is often established as a sentence even if it is directly replaced. For example, in the case of the sentence “I bought Hitachi's TV”, the sentence is established even if “Hitachi” is replaced with “Toshiba” or “GE”. Even if “TV” is replaced with “PC”, “mobile phone”, “smartphone” or the like, the sentence can be established. Here, words such as “Hitachi”, “Toshiba”, and “GE” can be collected by the concept of “company name”, for example. In addition, the words “TV”, “PC”, “mobile phone”, or “smart phone” can be aggregated under the concept of “product”, for example. This is especially true for words that are similar to nouns. Since most new words to be registered, such as words used uniquely in the industry or in the company, belong to nouns, the concept of aggregation based on this concept is considered particularly effective.

Specifically, as preprocessing, a “concept definition dictionary” for defining concepts, a “concept dictionary” for linking concepts and words, and a “concept relationship definition dictionary” for defining relationships between concepts Prepare. Using various dictionaries related to these concepts and a general-purpose morphological analysis dictionary existing in the market, the occurrence cost and the concatenation cost in this concept are calculated, and generated as a concept occurrence cost dictionary and a concept concatenation cost dictionary, respectively. Next, the occurrence cost and the concatenation cost are identified by using the concept occurrence cost dictionary and the concept concatenation cost dictionary for the words for which the occurrence cost and the concatenation cost are not set. Based on this identification result, a morphological analysis dictionary is reconstructed.
Thereafter, when morphological analysis is executed as the main process, the pre-processed morphological analysis dictionary is used.
As a result, even if the word occurrence cost and the concatenation cost are not set in advance, the word occurrence cost and the concatenation cost are automatically set from the concept to which the word belongs. Can be realized.

Specifically, the information analysis apparatus according to the present invention is an apparatus for analyzing text data composed of a natural language,
A word dictionary registered in association with a word used in a natural language and its characteristics, a concept dictionary in which a meaning tag indicating the meaning of the word and a meaning concept indicating the concept of the meaning tag are registered in association with each meaning of the word An externally defined dictionary acquisition means for reading dictionary data consisting of:
Based on the dictionary data acquired by the external definition dictionary acquisition means, concept characteristic estimation means for estimating the characteristics of the semantic concept by a characteristic set of words included in the semantic concept of the text data,
A concept dictionary in which a semantic tag indicating the meaning of a word additionally created by a user and a semantic concept indicating the concept of the semantic tag are registered in association with each meaning of the word is read and obtained by the concept characteristic estimation means Word characteristic estimation means for estimating the characteristic content of a word by associating it with the characteristic content of a semantic concept;
Analysis dictionary construction means for constructing an analysis dictionary by analyzing in an appropriate format for performing information analysis based on dictionary data obtained by the external definition dictionary acquisition means and word characteristics obtained by the word characteristic estimation means When,
Character information accepting means for accepting text information composed of a natural language input by the user as an input sentence;
A word extraction unit that identifies and extracts a word included in the input sentence obtained by the character information reception unit using the analysis dictionary generated by the analysis dictionary construction unit;
And an analysis result output means for selecting and outputting the extraction result obtained by the word extraction means based on a user instruction.

  In addition, the external definition dictionary acquisition means, as an external definition dictionary, a word dictionary registered in association with a word used in natural language and its characteristics, a meaning tag indicating the meaning of the word, and a meaning indicating the concept of the meaning tag In addition to a concept dictionary registered in association with each word meaning, the concept further includes means for reading a concept relation dictionary that defines the relationship between the semantic concepts, and the concept characteristic estimation means includes the semantic concept It further has a conceptual characteristic correcting means for correcting the conceptual characteristic in consideration of the relationship such as the vertical relationship between them.

Moreover, the improvement candidate holding means for holding in the storage device the items determined by the system that the result output by the analysis result output means could not be analyzed sufficiently,
It further comprises improvement candidate data presenting means for presenting improvement candidate data in accordance with an instruction of a user who maintains each dictionary.

The analysis method according to the present invention is a method of analyzing text data consisting of natural language by a computer,
The calculator is
A word dictionary registered in association with a word used in a natural language and its characteristics, a concept dictionary in which a meaning tag indicating the meaning of the word and a meaning concept indicating the concept of the meaning tag are registered in association with each meaning of the word A first step of reading dictionary data comprising:
A second step of estimating the characteristics of the semantic concept based on the characteristic set of words included in the semantic concept of the text data based on the dictionary data read in the first step;
A concept dictionary in which a meaning tag indicating the meaning of an additional word created by a user and a meaning concept indicating the concept of the meaning tag are registered in association with each meaning of the word is read and obtained in the second step A third step of estimating the characteristic content of the word by associating with the characteristic content of the semantic concept;
A third step of constructing an analysis dictionary by analyzing in an appropriate format for performing information analysis based on the dictionary data read in the first step and the characteristics of the words obtained in the second step When,
A fourth step of accepting text information consisting of a natural language input by the user as an input sentence;
A fifth step of identifying and extracting words contained in the input sentence received in the fourth step using the analysis dictionary constructed in the third step;
And a sixth step of selecting and outputting the extraction result extracted in the fifth step based on a user instruction.

  The first step includes a word dictionary registered in association with a word used in a natural language and its characteristics as an external definition dictionary, a semantic tag indicating the meaning of the word, and a semantic concept indicating the concept of the semantic tag. In addition to a concept dictionary registered in association with each meaning of the word, and a step of reading a concept relation dictionary defining the relationship between the semantic concepts, and the second step includes The method further includes a step of correcting the concept characteristic in consideration of the relationship such as the upper and lower relationship.

A step of holding the item determined by the system as a result of the output in the sixth step as a candidate for improvement in the storage device;
The method further includes the step of presenting the data for improvement according to an instruction of a user who maintains each dictionary.

Furthermore, in the first step, as an external definition dictionary, a word dictionary registered in association with a word used in a natural language and its characteristics, a semantic tag indicating the meaning of the word, and a semantic concept indicating the concept of the semantic tag In addition to the concept dictionary registered in association with each word meaning, and reading the concept relationship dictionary defining the relationship between the semantic concepts, and storing in the memory,
And a step of modifying concept characteristics in consideration of a relationship such as a vertical relationship between semantic concepts.

The program according to the present invention is a program for analyzing text data composed of natural language by a computer,
The calculator
A word dictionary registered in association with a word used in a natural language and its characteristics, a concept dictionary in which a meaning tag indicating the meaning of the word and a meaning concept indicating the concept of the meaning tag are registered in association with each meaning of the word An externally defined dictionary acquisition means for reading dictionary data consisting of:
Based on the dictionary data acquired by the external definition dictionary acquisition means, concept characteristic estimation means for estimating the characteristics of the semantic concept by a characteristic set of words included in the semantic concept of the text data,
A concept dictionary in which a semantic tag indicating the meaning of a word additionally created by a user and a semantic concept indicating the concept of the semantic tag are registered in association with each meaning of the word is read and obtained by the concept characteristic estimation means Word characteristic estimation means for estimating the characteristic content of a word by associating it with the characteristic content of a semantic concept;
Analysis dictionary construction means for constructing an analysis dictionary by analyzing in an appropriate format for performing information analysis based on dictionary data obtained by the external definition dictionary acquisition means and word characteristics obtained by the word characteristic estimation means When,
Character information accepting means for accepting text information composed of a natural language input by the user as an input sentence;
Word extraction means for identifying and extracting words included in the input sentence obtained by the character information reception means using the analysis dictionary constructed by the analysis dictionary construction means;
The extraction result obtained by the word extraction means is selected and output based on a user instruction, and functions as an analysis result output means.

  Further, the external definition dictionary acquisition means as an external definition dictionary, a word dictionary registered in association with a word used in a natural language and its characteristics, a meaning tag indicating the meaning of the word, and a meaning indicating the concept of the meaning tag In addition to the concept dictionary registered in association with each meaning of the word, the concept functions as a means for reading a concept relation dictionary defining the relationship between the semantic concepts, and the concept characteristic estimation means It is characterized by functioning as a conceptual characteristic correcting means for correcting the conceptual characteristic in consideration of the relationship such as the vertical relationship between them.

  In addition, the result output by the analysis result output unit functions as an improvement candidate holding unit that holds in the storage device the items that the system has determined to have not been sufficiently analyzed,

  According to an instruction from a user who maintains each dictionary, the dictionary is made to function as improvement candidate data presentation means for presenting improvement candidate data.

According to the present invention, when a dictionary maintenance worker registers a new word by providing a mechanism that considers the semantic concept of a word (vocabulary), it is only necessary to associate the concept with the word. In the first place, it is difficult for a person to calculate an occurrence cost and a connection cost related to a word, but it is quite easy to imagine which concept (semantic group) the word belongs to.
Thus, the dictionary maintenance cost in the morphological analysis dictionary can be reduced. Further, since new words can be easily registered in the dictionary itself, frequent maintenance can be performed, and as a result, the morphological analysis accuracy can be improved.

1 is a block diagram showing an overall configuration of an information analysis system in a first embodiment of the present invention. It is a block diagram which shows the structure of the information analysis dictionary construction subsystem 101 in the 1st Embodiment of this invention. It is explanatory drawing of the example of the word dictionary D110 in the 1st Embodiment of this invention. It is explanatory drawing of the example of the connection dictionary D120 in the 1st Embodiment of this invention. It is explanatory drawing of the example of the concept definition dictionary D130 in the 1st Embodiment of this invention. It is explanatory drawing of the example of the concept dictionary D140 in the 1st Embodiment of this invention. It is explanatory drawing of the example of the concept occurrence cost dictionary D150 in the 1st Embodiment of this invention. It is explanatory drawing of the example of the concept connection cost dictionary D160 in the 1st Embodiment of this invention. It is a flowchart of the process performed by the information analysis dictionary construction control program P11 in the first embodiment of the present invention. It is a flowchart which shows the outline | summary of the process performed by the conceptual cost analysis program P111 in the 1st Embodiment of this invention. It is a flowchart of the internal process (step S10704) performed by the conceptual cost analysis program P111 in the 1st Embodiment of this invention. It is explanatory drawing of the example of the concept occurrence cost model in the 1st Embodiment of this invention. It is explanatory drawing of the example of the concept connection cost model in the 1st Embodiment of this invention. It is explanatory drawing of the example of the unprocessed word model in the 1st Embodiment of this invention. It is a flowchart of the internal process (step S10707) performed by the conceptual cost analysis program P111 in the 1st Embodiment of this invention. It is a flowchart of the internal process (step S107071) performed by the conceptual cost analysis program P111 in the 1st Embodiment of this invention. It is a flowchart of the internal process (step S107072) performed by the conceptual cost analysis program P111 in the 1st Embodiment of this invention. It is a flowchart of the internal process (step S107073) performed by the conceptual cost analysis program P111 in the 1st Embodiment of this invention. It is a flowchart of the internal process (step S107074) performed by the conceptual cost analysis program P111 in the 1st Embodiment of this invention. It is explanatory drawing of the example of an image of the concept definition dictionary and concept dictionary in the 1st Embodiment of this invention. It is explanatory drawing of the example of a concept definition dictionary which added the word newly in the 1st Embodiment of this invention, and the image of a concept dictionary. It is explanatory drawing of the example of the word registration screen in the 1st Embodiment of this invention. It is a block diagram which shows the structure of the information analysis server 102 in the 1st Embodiment of this invention. It is a flowchart of the process performed by the information analysis program P12 in the 1st Embodiment of this invention. It is explanatory drawing of the example of the unknown word list | wrist D200 in the 1st Embodiment of this invention. It is a block diagram which shows the structure of the information analysis client 103 or 104 in the 1st Embodiment of this invention. It is a flowchart which shows the process performed by the analysis client control program P13 in the 1st Embodiment of this invention. It is a block diagram which shows the structure of the information analysis dictionary construction subsystem 101 in the 2nd Embodiment of this invention. It is a flowchart of the process performed by the information analysis dictionary construction control program P11 in the second embodiment of the present invention. It is explanatory drawing of the example of the conceptual relationship definition dictionary D170 in the 2nd Embodiment of this invention. It is a flowchart of the process (step S111) performed by the conceptual relationship adjustment program P112 in the 2nd Embodiment of this invention. It is explanatory drawing of the specific example of the concept definition dictionary D130 in the 2nd Embodiment of this invention. It is explanatory drawing of the specific example of the concept dictionary D140 in the 2nd Embodiment of this invention. It is explanatory drawing of the specific example of the conceptual relationship definition dictionary D170 in the 2nd Embodiment of this invention. It is explanatory drawing of the example of an image of the concept definition dictionary and concept dictionary in the 2nd Embodiment of this invention. It is an example of the lattice structure in a general morphological analysis engine.

Hereinafter, the present invention will be described in detail based on the illustrated embodiments.
<First Embodiment>
FIG. 1 is an overall configuration diagram of a system showing a first embodiment of the present invention.
As shown in FIG. 1, the information analysis system of this embodiment includes an information analysis dictionary construction subsystem 101, an information analysis server 102, and information analysis clients 103 and 104. Each of these devices is connected by a network 105.
The information analysis dictionary construction subsystem 101 is a computer for reconstructing a word dictionary and a connection dictionary used for morphological analysis, and a detailed configuration thereof will be described later with reference to FIG. That is, the information analysis dictionary construction subsystem 101 holds a concept dictionary for defining a semantic concept of a word and a concept dictionary for associating a concept with a word, in addition to a word dictionary and a connection dictionary used for morphological analysis. The information analysis dictionary construction subsystem 101 uses these various dictionaries to calculate occurrence costs and concatenation costs at the concept level, and generates them as a concept occurrence cost dictionary and a concept connection cost dictionary, respectively.
Thereafter, the word dictionary and the concatenation dictionary are reconstructed based on the concept occurrence cost dictionary and the concept concatenation cost dictionary.

  The reconstructed word dictionary and connection dictionary are transferred to the information analysis server 102 via the network 105 and used later when the information analysis server 102 performs analysis processing.

  The information analysis server 102 is a computer for analyzing information, and a detailed configuration thereof will be described later with reference to FIG. That is, the information analysis server 102 receives analysis commands from the information analysis clients 103 and 104, and analyzes data based on conditions specified by the analysis commands using various dictionary data obtained from the information analysis dictionary construction subsystem 101. The result data is sent back to the requesting information analysis client.

  The information analysis clients 103 and 104 are computers operated by a user who performs information analysis, and the detailed configuration thereof will be described later with reference to FIG. That is, the information analysis clients 103 and 104 display on the display a screen for the user to interactively specify analysis conditions, and the analysis conditions that can be interpreted by the information analysis server 102 on the screen are analyzed by the information analysis server 102. The data is converted into a command form, and the analysis command is transmitted to the information analysis server 102 via the network 105.

  As described above, when the information analysis server 102 processes the analysis command and returns the result data, the information analysis client displays the received result data on the screen and presents it to the user. Note that the user inputs the analysis conditions as follows: when the user himself / herself specifies a natural sentence from the keyboard, when using text data stored in the information analysis client, text data from a portable medium such as a floppy disk or optical disk May be used, or text data transferred via the network 105 may be used as input, but any form may be used.

  In addition, FIG. 1 shows an example in which two computers 103 and 104 are used as an information analysis client. However, the information analysis client may have a configuration of only one or three or more. You can also.

  The network 105 is a local area network (LAN) and / or a wide area network (WAN), and the information analysis dictionary subsystem 101, the information analysis server 102, and the information analysis clients 103 and 104 transmit and receive various data and commands. Used for.

  In FIG. 1, the network 105 is used to transfer various dictionary data from the information analysis dictionary construction subsystem 101 to the information analysis server 102. Instead, a portable medium such as a floppy disk or an optical disk is used. It can also take the structure to be used. Alternatively, the information analysis dictionary construction subsystem 101 and the information analysis server 102 may be mounted on a single computer and no data transfer may be performed.

  Furthermore, in FIG. 1, the information analysis clients 103 and 104 and the information analysis server 102 are assumed to use separate computers. However, one or more information analysis clients are executed on the same computer as the information analysis server. It can also be taken.

<Information analysis dictionary construction subsystem 101>
Next, the information analysis dictionary construction subsystem 101 in this embodiment will be described.
FIG. 2 is a block diagram showing the configuration of the information analysis dictionary construction subsystem 101 in this embodiment.
The information analysis dictionary construction subsystem 101 includes a memory 110, a storage device 120, a CPU (Central Processing Unit) 130, an output device 140, an input device 150, and a communication interface 160. Computer with a typical configuration.
The memory 110 stores a program executed by the CPU 130. Specifically, the system control program P10 and the information analysis dictionary construction control program P11 are stored in the memory 110. The memory 110 is provided with a work area for temporarily storing data when the CPU 130 executes the program.

The system control program P10 is a so-called operating system, and controls the entire information analysis dictionary construction subsystem 101.
The information analysis dictionary construction control program P11 is a program for constructing a dictionary necessary for information analysis, and includes a conceptual cost analysis program P111.
The conceptual cost analysis program P111 reads various dictionaries existing in the storage device 120, calculates a cost value at a conceptual level necessary for identifying a word occurrence cost and a concatenation cost, and stores the cost value in the storage device 120 ( Step S107 in FIG. 9) is a program.

The storage device 120 is a non-volatile storage element that can retain stored contents even when the power is shut off, and is configured by a magnetic disk drive (HDD) or a flash memory (SSD), for example. The storage device 120 stores various programs D100. The various programs D100 include the system control program P10 and the information analysis dictionary construction control program P11 described above, and are loaded into the memory 110 when executed by the CPU 130.
The storage device 120 also stores a word dictionary D110, a concatenation dictionary D120, a concept presentation dictionary D130, a concept dictionary D140, a concept occurrence cost dictionary D150, and a concept concatenation cost dictionary D160.
The word dictionary D110 is a word dictionary used for general morphological analysis processing, and holds word information as shown in FIG. As described above, in addition to the data shown in FIG. 3, a word ID, a part of speech, a utilization form, a word original form, a reading, and the like may be added.

The concatenation dictionary D120 is a concatenation dictionary used for general morpheme analysis processing, and holds concatenation information between context IDs related to words as shown in FIG.
The concept definition dictionary D130 holds information for associating an ID (identifier) for identifying a concept with a specific concept name as shown in FIG. In addition to the data shown in FIG. 5, other items such as the definition of the concept itself and the part of speech to which the concept belongs may be added.

The concept dictionary D140 holds information for associating a concept ID with a word, as shown in FIG. In addition to the items of concept ID and word, in addition to word information such as word ID, part of speech, inflection form, word original form, reading, etc., other items such as a weight value for linking the concept and word are added. It doesn't matter.
As shown in FIG. 7, the concept occurrence cost dictionary D150 holds information about the concept, in particular, occurrence cost information. The basic data structure is the same as that of the word dictionary D110, but an original item may be added.

  As shown in FIG. 8, the concept connection cost dictionary D160 holds information between context IDs related to concepts. The basic data structure is the same as that of the concatenation dictionary D120, but an original item may be added.

CPU 130 executes a program stored in memory 110.
The output device 140 is a display that displays the processing result on a screen, a printer that outputs the result to paper, or the like.
The input device 150 is a keyboard, a mouse, or the like for a user to give an instruction to the information analysis dictionary construction subsystem 101.

  The information analysis dictionary construction subsystem 101 stores various dictionaries obtained by executing the information analysis dictionary construction control program P11 in the storage device 120. Thereafter, the information is transferred to the information analysis server 102 via the network 105 according to the request from the information analysis server 102. Next, details of this processing procedure will be described.

FIG. 9 is a flowchart of processing executed by the information analysis dictionary construction subsystem 101 according to the first embodiment of this invention.
First, when a request from the user or the information analysis server 102 is received (step S101), the received request is analyzed (step S202).
If the received request is a dictionary construction command, information is stored in the storage device 120 according to the requested registration content.
Specifically, if it is a word dictionary registration request, the received word dictionary information is stored in the word dictionary storage area D110 (step S103). If it is a connection dictionary registration request, the received connection dictionary information is stored in the connection dictionary storage area D120 (step S104). If it is a concept definition dictionary registration request, the received concept definition dictionary information is stored in the concept definition dictionary storage area D130 (step S105). If it is a concept dictionary registration request, the received concept dictionary information is stored in the concept dictionary storage area D140 (step S106). If it is a request for calculating a concept cost, the concept cost generation program P111 is executed to generate a concept cost based on the word dictionary D110, the concatenation dictionary D120, the concept definition dictionary D130, and the concept dictionary D140 existing in the storage device 120. The cost and the connection cost are calculated and stored in the concept occurrence cost dictionary storage area D150 and the concept connection cost dictionary storage area D160 (step S107).

Details of step S107, that is, the conceptual cost analysis program P111 will be described later with reference to FIG. After the processing from step S103 to step S107 is completed, the process returns to step S101 to wait for reception of another request.
On the other hand, when the received request is an information transfer command, information corresponding to the requested inquiry content is read from the storage device 120 (step S108), and the read information is transmitted to the request source (step S109). Thereafter, the process returns to step S101 and waits for reception of another request.

On the other hand, when the received request is a subsystem stop command, the operation of the information analysis dictionary construction subsystem 101 is terminated.
Here, the details of the conceptual cost analysis program P111, that is, step S107 in FIG. 9, will be described with reference to the drawings.

FIG. 10 is a flowchart of processing executed by the conceptual cost analysis program P111 in the first embodiment of the present invention.
First, the conceptual cost analysis program P111 acquires the conceptual dictionary D140 from the storage device 120 and stores it in the work area (step S10701). Thereafter, the parameter n for controlling the loop is initialized to 1 (step S10702).
Then, the nth concept information is read from the concept dictionary D140 stored in the work area (step S10703). The concept information refers to information in which a concept ID is associated with a word.

  Based on the read concept information, a model used for dictionary construction such as a concept occurrence cost model, a concept connection cost model, and an unprocessed word model is updated (step S10704). Details of the processing in step S10704 and the concept occurrence cost model, concept concatenation cost model, and unprocessed word model will be described later with reference to FIGS.

  Thereafter, it is determined whether or not the (n + 1) th concept information exists (step S10705). As a result, if the next (n + 1) th conceptual information exists, 1 is added to the parameter (step S10706), and the process returns to step S10703 to perform analysis on the next conceptual information.

  On the other hand, if there is no next (n + 1) th concept information, the analysis for all the concept information obtained in step S10701 has been completed, so various dictionaries are generated using the various models updated ( Step S10707), the conceptual cost analysis program P111 is terminated. Details of step S10707 will be described later with reference to FIGS.

The above is the processing outline of the conceptual cost analysis program P111. Here, the details of the processing in step S10704 and the processing in step S10707 will be described.
First, step S10704, that is, a model update process used for dictionary construction will be described with reference to the drawings.
FIG. 11 is a flowchart of the process executed in step S10704.
When the information associated with the concept ID and the word is input and step S10704 is executed, first, the word dictionary D110 is acquired from the storage device 120 and stored in the work area (step S1070401). Further, the connection dictionary D120 is acquired from the storage device 120 and stored in the work area (step S1070402). Thereafter, the parameter s for controlling the loop is initialized to 1 (step S107043).

Then, the sth word information is read from the word dictionary D110 stored in the work area (step S1070404).
It is determined whether or not the read sth word information matches the word that is the input value (step S1070405). If they match, the processing of step S1070406 and step S1070407 is executed. If they do not match, the processing from step S1070408 to step S1070410 is executed.
If it is determined in step S1070405 that the input value matches the word, the concept occurrence is based on the concept ID obtained as the input value and the left context ID, right context ID, and occurrence cost existing in the sth word information. The cost model is updated (added) (step S1070406).

FIG. 12 is a diagram illustrating an example of a data structure of a concept occurrence cost model.
The concept occurrence cost model exists in the work area, and as shown in FIG. 12, the ID for identifying the concept, the left context ID, the right context ID, and the occurrence cost can be stored. The data obtained is directly added to this model. Here, the concept occurrence cost model is assumed to exist in the work area. However, a dedicated area for storing the concept occurrence cost model is secured in the storage device 120, and the update process is executed there. It doesn't matter.

Thereafter, the concatenation dictionary D120 stored in the work area is scanned to obtain concatenation information that matches the left context ID and the right context ID existing in the sth word information. Based on the acquired connection information, the concept connection cost model is updated (added) (step S1070407).
FIG. 13 is a diagram illustrating an example of a data structure of the conceptual connection cost model.
The conceptual connection cost model exists in the work area, and as shown in FIG. 13, the left context ID, the right context ID, and the connection cost can be stored. In addition, when the left context ID and the right context ID are used as keys, a plurality of concatenated costs can be stored. The data obtained is directly added to this model. Here, it is assumed that the conceptual connected cost model exists in the work area. However, a dedicated area for storing the conceptual connected cost model is secured in the storage device 120, and update processing is executed there. It doesn't matter.
When the process of step S1070407 ends, the process of step S10704 ends.

On the other hand, if it is determined in step S1070405 that the input value does not match the word, it is confirmed whether or not the next word information exists in the word dictionary D110 stored in the work area (step S1070408).
As a result, if the next (s + 1) th word information exists, “1” is added to the parameter (step S1070409), and the process returns to step S1070404 to execute the process for the next word information.

  On the other hand, if the next (s + 1) th word information does not exist, the processing for all the word information obtained in step S1070401 is complete, and it is considered that the corresponding word information does not exist. The unprocessed word model is updated (added) based on the concept information (step S1070410).

FIG. 14 is a diagram illustrating an example of the data structure of the unprocessed word model.
The unprocessed word model exists in the work area, and as shown in FIG. 14, the word itself and the ID for identifying the concept can be stored in association with each other. The data obtained is directly added to this model. Since the basic data structure is the same as that of the concept dictionary shown in FIG. 6, items other than the items shown in FIG. 14 may be added in accordance with the data structure of the concept dictionary. Here, the unprocessed word model is assumed to exist in the work area. However, a dedicated area for storing the unprocessed word model is secured in the storage device 120, and update processing is executed there. It doesn't matter.
After the process of step S1070410 is completed, the process of step S10704 is terminated.
The above is the details of the processing in step S10704.

Next, step S10707, that is, processing for generating various dictionaries using a model will be described with reference to the drawings.
FIG. 15 is a flowchart showing an overview of the processing executed in step S10707.
In step S10707, first, a concept occurrence cost dictionary is generated (step S107071), and a concept concatenation cost dictionary is generated (step S107072). Thereafter, a word dictionary used for morphological analysis is generated (step S107073), and a concatenated dictionary used for morphological analysis is generated (step S107074). Note that details of steps S107071 to S107074 will be described later with reference to the drawings.
First, step S107071, that is, a concept generation cost dictionary generation process will be described with reference to the drawings.

FIG. 16 is a flowchart of the process executed in step S107071.
In step S107071, first, a concept occurrence cost model existing in the work area is acquired (step S10707101). Further, the concept definition dictionary D130 existing in the storage device 120 is acquired and stored in the work area (step S10707102).
Based on the concept ID existing in the concept occurrence cost model and the concept ID existing in the concept definition dictionary D130, a complete list of concept IDs to be constructed is created (step S10707103). Thereafter, the parameter i for controlling the loop is initialized to “1” (step S10707104).

  Then, the i-th concept ID is read out from the entire list of concept IDs to be constructed (step S10707105). Based on the read concept ID, the concept occurrence cost model is scanned to determine the left context ID in the concept (step S10707106). This left context ID may be determined by setting the left context ID that is most frequent in the concept occurrence cost model corresponding to the concept ID. Similarly, the right context ID in the concept is determined (step S10707107).

  Next, the concept occurrence cost model is scanned based on the read concept ID, and the occurrence cost in the concept is calculated (step S10707108). In calculating the occurrence cost, an average value of the occurrence cost in the concept occurrence cost model corresponding to the concept ID may be set, or the mode value of the occurrence cost in the concept occurrence cost model corresponding to the concept ID is set. It may be set.

When the occurrence cost calculation in steps S10707106 to S10707109 is completed, the data of the concept occurrence cost dictionary D150 existing in the storage device 120 is updated using the concept ID as a key (step S10707109).
Thereafter, it is confirmed whether or not the next concept ID exists in the entire list of concept IDs to be constructed (S10707110).
As a result, if the next (i + 1) th information exists, “1” is added to the parameter (step S10707111), the process returns to step S10707105, and the process for the next word information is executed.

On the other hand, if the next (i + 1) th information does not exist, it is considered that the processing for all the conceptual information constructed in step S10707103 has been completed, and thus the processing in step S107071 is terminated.
The above is the detailed description of step S107071.

Next, step S107072, that is, a concept connection cost dictionary generation process will be described with reference to the drawings.
FIG. 17 is a flowchart of the process executed in step S107072.
In step S107072, first, a concept connection cost model existing in the work area is acquired (step S10707201). Further, the concept occurrence cost dictionary D150 existing in the storage device 120 is acquired and stored in the work area (step S10707202).
Full list of left context ID and right context ID pairs to be constructed based on the left context ID, right context ID, and left context ID, right context ID present in the concept occurrence cost dictionary D150 existing in the concept concatenation cost model Is created (step S10707203).
Thereafter, the parameter i for controlling the loop is initialized to “1” (step S10707204).

  Then, the i-th left context ID / right context ID pair is read from the entire list of left context ID / right context ID pairs to be constructed (step S10707205). Based on the read left context ID and right context ID pair, the conceptual concatenation cost model is scanned to calculate the concatenation cost value for the left context ID and right context ID pair (step S10707106). For calculating the concatenated cost value in the left context ID and right context ID pair, an average value of concatenated costs in the set corresponding to the left context ID and right context ID pair in the conceptual concatenated cost model may be adopted. The mode value may be adopted.

When the connection cost calculation is completed, the data of the concept connection cost dictionary D160 existing in the storage device 120 is updated using the obtained left context ID and right context ID as keys (step S10707207).
Thereafter, it is confirmed whether or not the next pair exists in the entire list of left context ID and right context ID pairs to be constructed (S10707108).
As a result, if the next (i + 1) th information exists, “1” is added to the parameter (step S10707109), and the process returns to step S10707205 to execute the process for the next word information.

On the other hand, if the next (i + 1) th information does not exist, it is considered that the processing for all the lists constructed in step S10707203 has been completed, and thus the processing in step S107072 is terminated.
The above is the detailed description of step S107072.
Next, step S107073, that is, word dictionary generation processing used for morphological analysis will be described with reference to the drawings.

FIG. 18 is a flowchart of the process executed in step S107073.
In step S107073, first, the concept occurrence cost dictionary D150 existing in the storage device 120 is acquired and stored in the work area (step S10707301). Further, an unprocessed word model existing in the work area is acquired (step S10707302).
Thereafter, the parameter i for controlling the loop is initialized to “1” (step S10707303).
Then, the i-th word information is read from the unprocessed word model (step S10707304).

  Based on the concept ID present in the read word information, the concept occurrence cost dictionary D150 is scanned to obtain the left context ID, right context ID, and occurrence cost in the concept (step S10707305). Based on the acquired concept occurrence cost, the occurrence cost of the word is calculated (step S10707306). The calculated occurrence cost is stored in the work area by associating the word with the left context ID and the right context ID in the aforementioned concept. In calculating the word occurrence cost, there is a method that considers the character string length of the word in addition to the method of setting the acquired concept occurrence cost as it is.

For example, if both “Hitachi” and “Hitachi Solutions” have the same occurrence cost value, the one with the shorter string length tends to be prioritized for morphological analysis, and the longer the string length, Since there is a high possibility of being confirmed as a word, the longer the character string length, the lower the occurrence cost. Specifically, Equation 1 shown in the lower part of FIG. 14 may be used.
When the occurrence cost calculation in step S10707305 and step S10707306 is completed, it is confirmed whether or not the next unprocessed word exists in the entire list of unprocessed words to be constructed (S10707307).

As a result, if there is a next (i + 1) th unprocessed word, “1” is added to the parameter (step S10707308), the process returns to step S10707304, and the process for the next word information is executed.
On the other hand, if the next (i + 1) th unprocessed word does not exist, it is considered that the processing for all the unprocessed words has been completed, so the storage device is based on the processed data existing in the work area. The word dictionary storage area D110 existing in 120 is updated (step S10707309).
After all the update processes are completed, the process of step S107073 is terminated.
The above is the detailed description of step S107073.

Next, step S107074, that is, a process for generating a connected dictionary used for morphological analysis will be described with reference to the drawings.
FIG. 19 is a flowchart of processing executed in step S107074.
In step S107074, first, the concept concatenation cost dictionary D160 existing in the storage device 120 is acquired (step S10707401). Further, the connection dictionary D120 existing in the storage device 120 is acquired and stored in the work area (step S10707402).
Based on the left context ID and right context ID existing in the concept concatenation cost dictionary D160 and the left context ID and right context ID existing in the concatenation dictionary D120, a complete list of pairs of left context ID and right context ID to be constructed is obtained. It is created (step S10707403).

Thereafter, the parameter i for controlling the loop is initialized to “1” (step S10707404).
Then, the i-th left context ID / right context ID pair is read out from the entire list of left context ID / right context ID pairs to be constructed (step S10707405). Based on the read pair of the left context ID and the right context ID, the concept concatenation cost dictionary D160 and the concatenation dictionary D120 are scanned (step S10707406), and the concatenation cost value in the pair of the left context ID and the right context ID is calculated (step S10707406). S10707107).

  The calculation of the concatenated cost value in the left context ID / right context ID pair employs the maximum concatenated cost value of the left context ID / right context ID pair existing in the conceptual concatenation cost dictionary D160 and the concatenation dictionary D120. Alternatively, when the value of the pair exists in both the conceptual connection cost dictionary D160 and the connection dictionary D120, an average value may be adopted. The calculated concatenated cost is stored in the work area in association with the left context ID and right context ID pair.

When the connection cost calculation in step S10707406 and step S10707407 is completed, it is confirmed whether or not the next pair exists in the entire list of left context ID and right context ID pairs to be constructed (S10707408).
As a result, if there is a next (i + 1) th pair, “1” is added to the parameter (step S10707409), the process returns to step S10707405, and the process for the next pair is executed.

On the other hand, if the next (i + 1) th pair does not exist, it is considered that the processing for all the left context ID and right context ID pairs to be constructed has been completed. Based on the data, the concatenated dictionary storage area D120 existing in the storage device 120 is updated (step S10707410).
After all the update processes are finished, the process of step S107074 is finished.
The above is the detailed description of step S107074.

The above is an explanation of the system configuration and processing overview of the information analysis dictionary construction subsystem 101 according to the first embodiment of the present invention.
Hereinafter, the information analysis dictionary construction process in the first embodiment will be described using a specific example.

<Specific example of information analysis dictionary construction>
In order to register a word in the dictionary for executing morphological analysis, it is confirmed whether or not the word exists in the dictionary, and if it does not exist, a new word is registered. In this embodiment, measures for reducing the confirmation work will be described later, but here, the word “Kyoto University” does not exist in the dictionary for executing morphological analysis, and it is desired to newly register it. A case will be described as an example.
As described above, normally, the word “Kyoto University” is added to the word dictionary as shown in FIG. 3, and the values of the left context ID, the right context ID, and the occurrence cost are not repeatedly set through trial and error. I didn't. Further, when a new ID to be described as the left context ID or the right context ID is created, the combination of corresponding IDs and the connection cost thereof are not set by trial and error in the connection dictionary as shown in FIG. I didn't. Therefore, there has been a problem that the cost for maintenance increases.
In this embodiment, it shows that the cost by repeating the trial and error becomes unnecessary.

First, a word dictionary as shown in FIG. 3 is prepared and registered. Specifically, the process of step S103 in FIG. 9 is executed. As this word dictionary, a dictionary generally used in morphological analysis may be used as it is. Here, it is assumed that the word “Kyoto University” is not included in the word dictionary.
Next, a connected dictionary as shown in FIG. 4 is prepared and registered. Specifically, the process of step S104 in FIG. 9 is executed. As for this concatenated dictionary, a dictionary generally used in morphological analysis may be used as it is as in the word dictionary.

Next, a concept definition dictionary as shown in FIG. 5 is prepared and registered. Specifically, the process of step S105 in FIG. 9 is executed. The concept definition dictionary will be described later.
Next, a concept dictionary as shown in FIG. 6 is prepared and registered. Specifically, the process of step S106 in FIG. 9 is executed. The concept dictionary will be described later.
The concept definition dictionary and concept dictionary described above will be described below.

FIG. 20 is an image of the concept definition dictionary and the concept dictionary, that is, examples of FIGS. 5 and 6. In FIG. 20, the circled characters indicate the concept name, and the rectangles in FIG. 20 indicate the surface shape of the word.
In the example of FIG. 20, the word “Toshiba” belongs to the concept “company”, and the word “Hitachi” belongs to the concept “company” and the concept “region”. Similarly, the word “Kyoto University” belongs to the concept of “education institution” and the concept of “facility”, and the word “Kyoto” belongs to the concept of “region”.

  These concept definition dictionaries and concept dictionaries may be constructed manually by the user, WordNet presented in Non-Patent Literature 8 and Non-Patent Literature 9, or EDR presented in Non-Patent Literature 10. A generally available dictionary such as the above may be used. Alternatively, it may be constructed using the method presented in Patent Document 3.

Assuming that the concept definition dictionary and the concept dictionary are constructed as described above, a case where the word “Kyoto University” is to be registered will be described.
First, the word “Kyoto University” is added to the concept dictionary as shown in FIG. Then, considering what the meaning of the word “Kyoto University” has, the concept definition dictionary data structure as shown in FIG. 5 is looked down, and the corresponding one is linked to the word “Kyoto University”. Here, it is assumed that the word “Kyoto University” belongs to the concept of “education institution” and the concept of “facility”. 5 that the concept ID of the concept “education institution” is “120” and the concept ID of the concept “facility” is “520”. Therefore, the word “Kyoto University” and the concept ID “120” may be linked to the concept dictionary, and the word “Kyoto University” and the concept ID “520” may be linked.
FIG. 21 is an image of the result of linking the word “Kyoto University” to the state shown in FIG.

As shown in FIG. 21, it can be said that mapping a certain word and a concept expressing its meaning is relatively easy.
Further, although the process for adding a definition to the concept definition dictionary is not described here, a new concept may be defined here.
In order to make it easier for the user to register these dictionaries, it is possible to easily prepare a dedicated screen. FIG. 22 shows an example of a registration screen.
After these definitions are made and the concept definition dictionary and the concept dictionary are registered, an actual dictionary construction process is performed. Specifically, the process of step S107 in FIG. 9 is performed.

  Actually, the model necessary for dictionary construction is updated in step S10704 of FIG. 10 which is an internal process of step S107. Specifically, since the word “Kyoto University” is not included in the original word dictionary, it is stored in the unprocessed word model in step S1070410 in FIG. 11, which is an internal process in step S10704 in FIG. become. An example of this unprocessed word model is shown in FIG. As shown in FIG. 14, the word “Kyoto University” and the concept ID are stored in this model in a linked form.

Thereafter, actual dictionary construction is performed in step S10707 of FIG. 10 which is internal processing of step S107. Here, since the word “Kyoto University” is stored in the unprocessed word model, the process is performed in step S107073 in FIG. 15 which is an internal process in step S10707 in FIG.
Here, as shown in FIG. 14, it is assumed that the word “Kyoto University” is associated with the concept IDs “120” and “520”.

  In this case, first, in step S10707305 in FIG. 18 which is internal processing in step S107073 in FIG. 15, the concept occurrence cost dictionary D150 is scanned, and data with the concept ID “120” is acquired. From FIG. 7, it is assumed that the data whose concept ID is 120 has a left context ID of “011”, a right context ID of “011”, and an occurrence cost of “1000”.

Thereafter, the occurrence cost of “Kyoto University” is calculated in step S10707306 in FIG. As described above, since it is appropriate to consider that the occurrence cost is the same value to some extent for the same concept, a value corresponding to the occurrence cost of the concept to which the word belongs may be set. Here, it is assumed that the above-described “Formula 1” is used, and the calculation is performed when the weight α in the “Formula 1” is 4. Then, the numerical value of 1000− {4 × 2 ^ (1.3)} = 990.150... ≈990 is obtained from Equation 1. Therefore, the numerical value of 990 is set to the left context ID “011”, the right The occurrence cost of “Kyoto University” with the context ID “011” is assumed.
Similarly, data with the concept ID “520” is acquired, and the occurrence cost of “Kyoto University” with the left context ID “100” and the right context ID “100” is calculated.
As a result, the occurrence cost of “Kyoto University” is obtained, and the data is additionally updated in the word dictionary storage area D110.

After that, in steps S10707406 and S10707407 in FIG. 19, which is an internal process in step S107074 in FIG. 15, the concatenated cost with the left context ID “100” and the right context ID “100” is calculated and automatically The concatenated cost value is updated.
With this procedure, instead of setting by trial and error by the user, the user simply maps the word “Kyodai” and its meaning to the concept dictionary, and automatically adds the word “Kyodai” to the word dictionary. In addition, it is possible to automatically calculate and set appropriate values of the occurrence cost and the connection cost necessary for executing the morphological analysis.

In this example, the process up to execution of dictionary construction is illustrated on the assumption that a new concept dictionary is registered, but the concept dictionary storage area of the storage device 120 can be used even when a new word is additionally set in an existing concept dictionary. Since the current data exists in D140, the necessary information may be appropriately updated here to execute the corresponding process.
The above is the description of the information analysis dictionary construction process in the first embodiment.

<Information analysis server 102>
Next, the information analysis server 102 in this embodiment will be described.
FIG. 23 is a block diagram showing the configuration of the information analysis server 102 in this embodiment.
The information analysis server 102 has the same configuration as the information analysis dictionary construction subsystem 101 described above except that the stored program is different. For this reason, the same code | symbol is attached | subjected to the same structure as the information analysis dictionary construction subsystem 101 mentioned above, and the description is abbreviate | omitted.
That is, the information analysis server 102 includes a memory 110, a storage device 120, a CPU (Central Processing Unit) 130, an output device 140, an input device 150, and a communication interface 160. It is a computer with a simple configuration.

  The memory 110 stores a program executed by the CPU 130. Specifically, the system control program P10 and the information analysis program P12 are stored in the memory 110.

  The information analysis program P12 is a program for analyzing information based on the analysis request transmitted from the information analysis client 103 or 104, and includes a dictionary loading program P121, an analysis conditional expression analysis program P122, and a morpheme analysis program P123 as subprograms. And an unknown word list acquisition program P124.

The dictionary loading program P121 acquires the various dictionaries constructed by the information analysis dictionary construction subsystem 101 and stores them in the storage device 120 (step S203 and step S204 in FIG. 24).
The analysis conditional expression analysis program P122 analyzes the analysis conditional expression input by the user (step S205 in FIG. 24).
The morpheme analysis program P123 is a program that executes morpheme analysis (step S206 in FIG. 24).
The unknown word list acquisition program P124 transfers the contents of the unknown word list D200 existing in the storage device 120 to the request source (step S210 and step S211 in FIG. 24).

The storage device 120 stores various programs D100. The various programs D100 include a system control program P10 and an information analysis program P12, which are loaded into the memory 110 when executed by the CPU 130.
Further, the storage device 120 includes a word dictionary D110 and a concatenation dictionary D120. The word dictionary D110 and the connection dictionary D120 are dictionary data constructed by the information analysis dictionary construction subsystem 101, loaded from the information analysis dictionary construction subsystem 101 by the dictionary loading program P121, and stored in the storage device 120.

Further, the storage device 120 includes an unknown word list D200. The unknown word list D200 is a list of words (hereinafter referred to as unknown words) that could not be sufficiently analyzed as a result of analysis by the morphological analysis program P123, and is transferred to the request source by the unknown word list acquisition program P124.
By executing the information analysis program P12, the information analysis server 102 analyzes data based on the analysis request transmitted from the information analysis client 103 or 104, and returns the analysis result to the requesting information analysis client. Next, details of this processing will be described with reference to the drawings.

FIG. 24 is a flowchart of processing executed by the information analysis server 102 according to the first embodiment of this invention.
First, when the information analysis server 102 receives a request from the request source (step S201), the information analysis server 102 analyzes the received request (step S202).
If the received request is a dictionary loading request, the dictionary loading program P121 is executed to obtain the word dictionary and the concatenated dictionary stored in the information analysis dictionary construction subsystem 101. An information transfer command is issued (step S203), and various dictionary data returned from the information analysis dictionary construction subsystem 101 are stored in the word dictionary storage area D110 and the connected dictionary storage area D120 of the storage device 120.

On the other hand, if the received request is an unknown word list transfer request, the unknown word list D200 existing in the storage device 120 is acquired (step S210), and the unknown word list information acquired by the request source that made this request is obtained. Transfer (step S211).
On the other hand, when the received request is a server stop request, the operation of the information analysis server 102 is stopped.
On the other hand, if the received request is an analysis request, the analysis conditional expression analysis program P122 is executed to analyze the analysis conditional expression (step S205). Specifically, the analysis target text input by the user is acquired from the analysis conditional expression.

  Next, the morpheme analysis program P123 is executed, and the morpheme analysis process is executed using the word dictionary D110 and the concatenation dictionary D120 existing in the storage device 120 based on the analysis target text included in the analysis conditional expression (step S206). ). Note that details of the morphological analysis process are well known and are omitted. Further, the word dictionary D110 and the concatenated dictionary D120 existing in the storage device 120 are in the form of a dictionary for executing morphological analysis, so that no special method is required for the morphological analysis processing itself.

After execution of the morpheme analysis process in step S206, it is confirmed whether or not an unknown word is included in the morpheme set existing as an analysis result (step S207). If the unknown word does not exist, the process proceeds to step S209.
On the other hand, if an unknown word exists in the analysis result morpheme set, the unknown word set included in the analysis result is registered in the unknown word list D200 (step S208).
Thereafter, the analysis result obtained in step S206 is returned to the request source (step S209).

FIG. 25 is a diagram illustrating an example of the unknown word list D200.
As shown in FIG. 25, a list of words determined as unknown words as a result of the morphological analysis, that is, vocabularies not registered in the word dictionary is stored. In FIG. 25, only the word notation of the unknown word is stored, but for reference information when performing dictionary maintenance, the number of appearances, the character strings before and after the unknown word appear, etc. Other items may be stored together.
Since the unknown word list D200 exists, it is clearly understood that the unknown word in the unknown word list is a word that does not exist in the dictionary. Therefore, an index for performing dictionary maintenance can be obtained. As a result, it is possible to greatly reduce the procedure of checking whether the word exists in the dictionary and registering a new word if it does not exist.
This completes the explanation of the information analysis server 102 in the first embodiment of the present invention.

<Information analysis clients 103 and 104>
Next, the information analysis clients 103 and 104 in this embodiment will be described.
FIG. 26 is a block diagram showing the configuration of the information analysis client 103 or 104 in this embodiment.
The information analysis client 103 or 104 has the same configuration as the information analysis dictionary construction subsystem 101 (FIG. 2) described above except that the stored program is different. For this reason, the same code | symbol is attached | subjected to the same structure as the information analysis dictionary construction subsystem 101 mentioned above, and the description is abbreviate | omitted.
That is, the information analysis client 103 or 104 includes a memory 110, a storage device 120, a CPU (Central Processing Unit) 130, an output device 140, an input device 150, and a communication interface 160, and these components are connected by a bus 170. It is a computer.
The memory 110 stores a program executed by the CPU 130. Specifically, the system control program P10 and the analysis client control program P13 are stored in the memory 110.

The analysis client control program P13 is a program that generates an analysis request to be transmitted to the information analysis server 102, and includes an analysis condition input program P131 and an analysis result display program P132 as subprograms.
The analysis condition input program P131 accepts input of analysis conditions from the user and transmits an analysis request to the information analysis server 102. The analysis result display program P132 displays the analysis result in accordance with an instruction from the user.
The storage device 120 stores various programs D100. The various programs D100 include a system control program P10 and an analysis client control program P13, which are loaded into the memory 110 when executed by the CPU 130.

The storage device 120 stores analysis result data D300. The analysis result data D300 is a cache in which the analysis result transferred from the information analysis server 102 is temporarily stored.
The information analysis client 103 or 104 generates an analysis request to be transmitted to the information analysis server 102 by executing the analysis client control program P13, and displays the result of the analysis performed by the information analysis server 102. Next, details of this processing will be described with reference to the drawings.

FIG. 27 is a flowchart of processing executed by the information analysis client 103 or 104 according to the first embodiment of this invention.
When the analysis client control program P13 is started, first, an analysis condition input screen is displayed to prompt input of an instruction (command) (step S301).
Thereafter, when a command is input (step S302), the input command is analyzed (step S303).
If the analyzed command is a client stop command, the analysis client control program P13 is terminated.

On the other hand, if the analyzed command is an analysis command, the analysis condition input program P131 is executed, an analysis request is generated based on the input data (step S304), and the generated analysis request is transmitted to the information analysis server 102. (Step S305).
Thereafter, when an analysis result is received from the information analysis server 102, the received analysis result is stored in the analysis result data D300 of the storage device 120 (step S306). The analysis result includes a list of word information obtained from the text to be analyzed, and is transmitted from the information analysis server 102 in step S209 in FIG.

  Thereafter, the analysis result display program P132 is activated to display an analysis result display screen (step S307). At this time, a screen for inputting the next command is also displayed. Then, it returns to step S302 and receives a command input from the user.

As described above, this embodiment provides a mechanism that takes into account the semantic definition of a word when a dictionary for morphological analysis is constructed. For this reason, even if a word is to be added to the dictionary, the user who maintains the dictionary only needs to map the word to be added and the concept thereof, and it is necessary to make settings by trial and error by a user who is expensive in constructing the dictionary. Therefore, the maintenance cost of the dictionary can be reduced.
Furthermore, in this embodiment, since words (unknown words) that could not be sufficiently analyzed as a result of actual analysis can be accumulated, the user determines the next word to be registered in the dictionary using the word. Will be able to. That is, it becomes possible to actively maintain the dictionary by acquiring unknown words and registering new words easily, and as a result, it is possible to improve analysis accuracy.

<Second embodiment>
Next, a second embodiment of the present invention will be described with reference to the drawings. In the second embodiment, the system configuration is the same as that of the first embodiment, but the method of generating the concept occurrence cost dictionary is different in the processing of the information analysis dictionary construction subsystem 101. As a result, the word The starting cost value in the dictionary will be different.

  In the first embodiment, in order to generate a concept occurrence cost dictionary, a concept occurrence cost is calculated using only information that links a word set by a user and a concept (FIG. 9). Step S107 and its internal processing, step S10704 in FIG. 10). However, when the number of words associated with the concept is small, the tendency due to the small number of words is calculated as it is as the cost of occurrence of the concept. In addition, it is not possible to calculate the occurrence cost of a concept that is not associated with a word or a concept that is associated with only a word that does not have a word occurrence cost even if it is associated with a word. In such a system, with the passage of time, it is assumed that it would be desirable to link even a concept that was not originally linked to a word, and a sufficient concept generation cost is estimated only by the first embodiment. As a result, it may be inadequate for estimating the cost of occurrence of words.

In the second embodiment, in addition to the concept cost analysis program P111 in the first embodiment, attention is paid to the relationship between the concepts, and the occurrence cost of the concept adjusted based on the relationship is calculated. It is characterized in that the occurrence cost estimation accuracy is improved.
Details of the information analysis dictionary construction subsystem 101 in this embodiment will be described below.

<Information analysis dictionary construction subsystem 101>
In addition, the same code | symbol is attached | subjected to the same structure as the information analysis dictionary construction subsystem 101 (FIG. 2) in 1st Embodiment, and the description is abbreviate | omitted.
FIG. 28 is a diagram showing a system configuration of the information analysis dictionary construction subsystem 101 in the present embodiment.
The information analysis dictionary construction subsystem 101 shown in FIG. 28 has the same hardware configuration as that of the first embodiment shown in FIG. However, in the memory 110, in addition to the program group held in the first embodiment, the conceptual relationship adjustment program P112 is held. Further, in the storage device 120, a concept relation definition dictionary D170 is secured in addition to the area group secured in the first embodiment.
The concept relationship adjustment program P112 is a program that adjusts the occurrence cost in the concept based on the relationship between the concepts and updates the occurrence cost of the word (step S111 in FIG. 29).

  The concept relationship definition dictionary D170 is information for defining the relationship between concepts such as vertical relationships. If you think that a concept is almost an agreement with the word type, for example, there is a higher-level concept, a concept that is part of it, or a concept that has the opposite meaning. It is considered normal. The concept relationship definition dictionary D170 is used to define the relationship between such concepts. As shown in FIG. 30, the concept relationship definition dictionary D170 holds information for associating an ID (identifier) pair for identifying a concept with its relationship (role).

In addition to the data items shown in FIG. 30, other items such as the definition of the role itself may be added. In addition to the tree structure, the conceptual relationship structure may take any form such as a network graph structure such as a bipartite graph.
Further, the conceptual relationship definition dictionary D170 may be manually constructed by the user, as in the conceptual definition dictionary D130 and the conceptual dictionary D140 in the first embodiment, or presented in Non-Patent Document 8 and Non-Patent Document 9. Generally available dictionaries such as WordNet, or EDR presented in Non-Patent Document 10 may be used. Alternatively, it may be constructed using the method presented in Patent Document 3.

FIG. 29 is a flowchart of processing executed by the information analysis dictionary construction subsystem 101 according to the second embodiment of this invention.
When compared with the flowchart of the process executed by the information analysis dictionary construction subsystem 101 in the first embodiment, that is, when compared with FIG. 9, the process contents are the same except that steps S110 and S11 1 are added. For this reason, the same processing contents as those of the information analysis dictionary construction subsystem 101 described above are denoted by the same reference numerals and description thereof is omitted.

First, when a request from the user or the information analysis server 102 is received (step S101), the received request is analyzed (step S202).
If the received request is a dictionary construction command, information is stored in the storage device 120 according to the requested registration content. Specifically, the content is the same as that described in the first embodiment, but is stored in the storage device 120 based on the conceptual relationship definition dictionary registration command as compared to the first embodiment (FIG. 29). Step S110) Processing is added.

Further, after step S107 in FIG. 9 or FIG. 29 is completed, the process of step S111 is executed. That is, when step S107 is completed, the word dictionary D110, the concatenation dictionary D120, the concept definition dictionary D130, the concept dictionary D140, the concept occurrence cost dictionary D150, and the concept concatenation cost dictionary D160 in FIG. 2 or 28 are once constructed. Think of things.
Here, details of step S111 in FIG. 29 will be described with reference to the drawings.

FIG. 31 is a flowchart of processing executed in step S111 of FIG. 29, that is, the conceptual relationship adjustment program P112.
First, the conceptual relationship adjustment program P112 first reads the conceptual relationship definition dictionary D170 existing in the storage device 120, and stores the entire list of conceptual relationship definitions in the work area (step S11101). Further, all data on the concept occurrence cost dictionary D150 is read from the storage device 120 and stored in the work area as a concept occurrence cost model. Similarly, all data on the concept connection cost dictionary D160 is read from the storage device 120 and stored in the work area as a concept connection cost model. (Step S11102).

The concept generation cost model is the model described in the first embodiment, and is the same as that in FIG. 12 in the first embodiment. The conceptual connection cost model is the model described in the first embodiment, and is the same as FIG. 13 in the first embodiment. Thereafter, the initial value “1” is set for the parameters a and b for controlling the loop (step S11103).
Then, the a-th concept occurrence cost information is acquired from the concept occurrence cost model stored in the work area (step S11104).

Next, b-th concept definition information is acquired from the entire list of concept relationship definitions stored in the work area (step S11105). In this conceptual relationship definition, as shown in FIG. 30, two IDs for identifying the concepts are defined, and information indicating these relationships (roles) is stored.
Based on the a-th concept occurrence cost information and the b-th concept definition information, whether or not the concept ID existing in the a-th concept occurrence cost information matches any concept ID existing in the b-th concept definition information Is determined (step S11106), and if they match, the process proceeds to step S11107 and step S11108.

On the other hand, if it is determined in step S11106 that they do not match, it is determined whether or not the next (b + 1) th concept definition information exists (step S11109). If the next conceptual information exists, “1” is added to the parameter b, and the process returns to step S11105. If the next concept definition information does not exist, it is determined that all concept information has been scanned, the parameter b is set to the initial state, that is, “1”, and the process proceeds to step S11112.
In step S11107, among the concept IDs present in the b-th concept definition information, the left existing in the a-th concept occurrence cost information together with the concept ID that does not match the concept ID present in the a-th concept occurrence cost information. The context ID, right context ID, and occurrence cost are set and added to the concept occurrence cost model.

Next, in step S11108, the concept connected cost model existing on the work area is scanned using the left context ID and the right context ID existing in the a-th concept occurrence cost information as keys, and the corresponding connected cost value is acquired. The left context ID and the right context ID existing in the acquired concatenated cost value and the a-th concept occurrence cost information are set and added to the concept concatenated cost model.
After the process of step S11108, the parameter b is set to the initial state, that is, “1” in step S11111, and the process proceeds to step S11112.

  In step S11112, it is confirmed whether the next (a + 1) th information exists in the concept occurrence cost model stored in the work area. If the (a + 1) th concept occurrence cost information exists, “1” is added to the parameter a (step S11113), and the process returns to step S11104.

On the other hand, if the (a + 1) th concept occurrence cost information does not exist, it is considered that all concept occurrence cost information has been scanned, and various dictionaries are generated from the generated concept occurrence cost model, concept concatenation cost model, and the like (step S11114). . Note that step S11114 is exactly the same as step S10704 in FIG. 10 in the first embodiment. Therefore, details of step S11114 are omitted.
When the process of step S11114 is finished, it is considered that all the processes are finished, and the conceptual relationship adjustment program P112, that is, the process of step S111 is finished.

<Specific example of information analysis dictionary construction>
Hereinafter, the information analysis dictionary construction process in the second embodiment will be described using a specific example.
Here, consider a case where abbreviations of department names in a company are registered in a dictionary. The department name is uniquely named for the company, and the abbreviation of the department name is based on the custom of the company, so it cannot exist in a general dictionary. Also, in the abbreviations of department names, kanji and katakana are often mixed.

Some general morphological analysis engines have a function of recognizing the same character type as much as possible. However, since the department name abbreviations contain multiple character types, the department name abbreviations cannot be analyzed well.
Here, as an example of a department name and its abbreviation in a certain company, “production technology center” which is an abbreviation of “production technology center” and “1 management” which is an abbreviation of “first sales headquarters” are considered. Both abbreviations include two or more character types. Therefore, when these words are analyzed by a general morphological analysis engine without being registered, they are often written in the form of “skills / se”, “1 / operating / book”.
Therefore, the words “raw technology” and “1 management” are registered in the dictionary. Basically, the procedure is the same as that described in the description of the first embodiment, but here, in order to explain in the second embodiment, it is considered that the concept of “department” is newly established.

  FIG. 32 is a specific example of the concept definition dictionary D130 when the concept of “department” is newly established. Here, as shown in FIG. 32, the concept “department” is registered with the concept ID “140”. In addition, the words “production technology” and “1 management” are defined in the concept dictionary D140 in association with the concept ID. Here, as shown in FIG. 33, it is assumed that the words “skills” and “1 management” are associated with the concept ID “140”, respectively.

  In the first embodiment, the occurrence cost of the concept is calculated with the above settings. At this point, the words associated with the concept of “department” are “skills” and “1 management”. Since these are newly registered words, no occurrence cost is set for those words. Therefore, in the first embodiment, it is difficult to calculate a reasonable occurrence cost in the concept. Therefore, in the second embodiment, a relationship between concepts, that is, a connection between meanings of words is newly used.

  FIG. 34 shows a specific example of the concept relationship definition dictionary D170 that defines the relationship between the concepts, that is, the specific example of FIG. From FIG. 34, for example, it can be seen that the one with the concept ID “100” is connected to the one with the concept ID “110” by the role of “upper”. Further, it can be seen from FIG. 32 that the concept ID of 100 is the concept of “organization” and the concept ID of “110” is “company”.

  That is, it can be seen that the concept of “organization” is a superordinate concept of the concept of “company”. Similarly, the concept “organization” is a superordinate concept of the concept “department”, and the concept “department” is a partial concept of the concept “company”. FIG. 35 is an image of the above. As shown in FIG. 35, it can be said that it is relatively easy to map a certain word and a concept expressing its meaning or to map a concept and a concept.

  In FIG. 34 or FIG. 35, only one relationship between concepts is defined, but a plurality of relationships between concepts can be defined or none can be defined. For example, it may be defined that “the concept of organization is a superordinate concept of the concept of company” and “the concept of enterprise is a subordinate concept of the concept of organization”.

Furthermore, as described in the first embodiment, it is possible to easily prepare a dedicated screen in order to improve the efficiency of the registration work of these dictionary information.
Consider the case where step S111 in FIG. 29 is executed based on these. Here, as the execution result of step S107 of FIG. 29, description is advanced on the assumption that the left context ID of the concept “company” is “011”, the right context ID is “011”, and the occurrence cost is “100”. Hereinafter, the description will be given based on FIG.

If the concept occurrence cost information extracted in step S11104 in FIG. 31 has the concept “company”, the concept “company”, that is, the concept ID includes “110” in steps S11105 to S11110. The conceptual relationship definition information is scanned.
At a certain timing, when the items of concept ID-1 “100” and concept ID-2 “110” are hit and their role is “higher”, the concept generation cost model is updated in step S11107. . Specifically, a value obtained by multiplying the occurrence cost “100” in the concept “company” by a certain weight value is positioned as the occurrence cost of the concept. A certain weight value may be set in advance for each role, such as “0.95” for the superordinate concept and “0.9” for the partial concept. Here, 100 × 0.95 = 95 is set.

Thereafter, the concept ID “100”, the left context ID “011”, the right context ID “011”, and the calculated occurrence cost “95” in the concept “company” are set and added to the concept occurrence cost model.
Also, at a certain timing, when the items of concept ID-1 “140” and concept ID-2 “110” are hit and the role is “partial”, the concept occurrence cost model is updated in step S11107. I do. Here, if the partial concept relationship is set to a weight of “0.9”, the occurrence cost of the concept ID “140” has a value of 100 × 0.9 = 90.
Therefore, the concept ID “140”, the left context ID “011”, the right context ID “011”, and the calculated occurrence cost “90” in the concept “company” are added to the concept occurrence cost model as a set.

  By repeating the above process, even in the case of a concept such as “organization” where no word is associated, or in the case where it is associated only with a word that does not have a word occurrence cost such as “department”, The cost of occurrence appropriate for the concept will be calculated. Note that the subsequent processing is the same as that in the first embodiment, and is therefore omitted.

  As described above, by applying this embodiment, it becomes possible to consider the relationship between meanings of words, that is, the relationship between concepts. In addition, even when a concept is newly defined and a word associated with the new concept is newly registered, an appropriate cost value can be calculated.

101 ... Information analysis dictionary construction subsystem,
102: Information analysis server,
103 and 104 ... Information analysis client,
105 ... Network,
110: Memory,
120... Storage device,
130: CPU (Central Processing Unit),
140 ... output device,
150 ... input device,
160 ... communication interface,
170 ... Bus

Claims (10)

A device for analyzing text data consisting of natural language,
A word dictionary registered in association with a word used in a natural language and its characteristics, a concept dictionary in which a meaning tag indicating the meaning of the word and a meaning concept indicating the concept of the meaning tag are registered in association with each meaning of the word An externally defined dictionary acquisition means for reading dictionary data consisting of:
Based on the dictionary data acquired by the external definition dictionary acquisition means, concept characteristic estimation means for estimating the characteristics of the semantic concept by a characteristic set of words included in the semantic concept of the text data,
A concept dictionary in which a semantic tag indicating the meaning of a word additionally created by a user and a semantic concept indicating the concept of the semantic tag are registered in association with each meaning of the word is read and obtained by the concept characteristic estimation means Word characteristic estimation means for estimating the characteristic content of a word by associating it with the characteristic content of a semantic concept;
Analysis dictionary construction means for constructing an analysis dictionary by analyzing in an appropriate format for performing information analysis based on dictionary data obtained by the external definition dictionary acquisition means and word characteristics obtained by the word characteristic estimation means When,
Character information accepting means for accepting text information composed of a natural language input by the user as an input sentence;
A word extraction unit that identifies and extracts a word included in the input sentence obtained by the character information reception unit using the analysis dictionary generated by the analysis dictionary construction unit;
An information analysis apparatus comprising: an analysis result output unit that selects and outputs an extraction result obtained by the word extraction unit based on a user instruction.   The external definition dictionary acquisition means, as an external definition dictionary, a word dictionary registered in association with a word used in natural language and its characteristics, a semantic tag indicating the meaning of the word, and a semantic concept indicating the concept of the semantic tag; In addition to the concept dictionary registered in association with each meaning of the word, and further includes means for reading a concept relation dictionary defining the relationship between the semantic concepts, and the concept characteristic estimation means includes The information analysis apparatus according to claim 1, further comprising conceptual characteristic correcting means for correcting the conceptual characteristics in consideration of a relationship such as a vertical relationship. Improvement candidate holding means for holding, in the storage device, items determined by the system that the result output by the analysis result output means could not be analyzed sufficiently;
The information analysis apparatus according to claim 1, further comprising improvement candidate data presenting means for presenting improvement candidate data in accordance with an instruction of a user who maintains each dictionary. A method of analyzing text data composed of natural language by a computer,
The calculator is
A word dictionary registered in association with a word used in a natural language and its characteristics, a concept dictionary in which a meaning tag indicating the meaning of the word and a meaning concept indicating the concept of the meaning tag are registered in association with each meaning of the word A first step of reading dictionary data comprising:
A second step of estimating the characteristics of the semantic concept based on the characteristic set of words included in the semantic concept of the text data based on the dictionary data read in the first step;
A concept dictionary in which a meaning tag indicating the meaning of an additional word created by a user and a meaning concept indicating the concept of the meaning tag are registered in association with each meaning of the word is read and obtained in the second step A third step of estimating the characteristic content of the word by associating with the characteristic content of the semantic concept;
A third step of constructing an analysis dictionary by analyzing in an appropriate format for performing information analysis based on the dictionary data read in the first step and the characteristics of the words obtained in the second step When,
A fourth step of accepting text information consisting of a natural language input by the user as an input sentence;
A fifth step of identifying and extracting words contained in the input sentence received in the fourth step using the analysis dictionary constructed in the third step;
And a sixth step of selecting and outputting the extraction result extracted in the fifth step based on a user instruction.   The first step includes, as an external definition dictionary, a word dictionary registered in association with a word used in a natural language and its characteristics, a semantic tag indicating the meaning of the word, and a semantic concept indicating the concept of the semantic tag. In addition to the concept dictionary registered in association with the meaning of each word, the method further includes a step of reading a concept relation dictionary that defines the relationship between the semantic concepts, and the second step includes an upper and lower order between semantic concepts. 5. The information analysis method according to claim 4, further comprising a step of correcting the concept characteristic in consideration of a relationship such as a relationship. Holding the item determined by the system as a result of the output in the sixth step as a candidate for improvement in the storage device;
The information analysis method according to claim 4, further comprising a step of presenting the improvement data in accordance with an instruction of a user who maintains each dictionary. In the first step, as an external definition dictionary, a word dictionary registered in association with a word used in a natural language and its characteristics, a semantic tag indicating the meaning of the word, and a semantic concept indicating the concept of the semantic tag Reading a concept relation dictionary defining the relationship between the semantic concepts in addition to the concept dictionary registered in association with each meaning of the word, and storing in the memory;
The information analysis method according to claim 5, further comprising a step of correcting concept characteristics in consideration of a relationship such as a vertical relationship between semantic concepts. A program for analyzing text data composed of natural language by a computer,
The calculator
A word dictionary registered in association with a word used in a natural language and its characteristics, a concept dictionary in which a meaning tag indicating the meaning of the word and a meaning concept indicating the concept of the meaning tag are registered in association with each meaning of the word An externally defined dictionary acquisition means for reading dictionary data consisting of:
Based on the dictionary data acquired by the external definition dictionary acquisition means, concept characteristic estimation means for estimating the characteristics of the semantic concept by a characteristic set of words included in the semantic concept of the text data,
A concept dictionary in which a semantic tag indicating the meaning of a word additionally created by a user and a semantic concept indicating the concept of the semantic tag are registered in association with each meaning of the word is read and obtained by the concept characteristic estimation means Word characteristic estimation means for estimating the characteristic content of a word by associating it with the characteristic content of a semantic concept;
Analysis dictionary construction means for constructing an analysis dictionary by analyzing in an appropriate format for performing information analysis based on dictionary data obtained by the external definition dictionary acquisition means and word characteristics obtained by the word characteristic estimation means When,
Character information accepting means for accepting text information composed of a natural language input by the user as an input sentence;
Word extraction means for identifying and extracting words included in the input sentence obtained by the character information reception means using the analysis dictionary constructed by the analysis dictionary construction means;
A program for causing an extraction result obtained by the word extraction means to function as an analysis result output means for selecting and outputting based on a user instruction.   The external definition dictionary acquisition means as an external definition dictionary, a word dictionary registered in association with a word used in a natural language and its characteristics, a semantic tag indicating the meaning of the word, and a semantic concept indicating the concept of the semantic tag; In addition to the concept dictionary registered in association with each meaning of the word, and function as means for reading the concept relation dictionary defining the relationship between the semantic concepts, and further, the concept characteristic estimation means, 9. The program according to claim 8, wherein the program is made to function as a conceptual characteristic correcting unit that corrects a conceptual characteristic in consideration of a relationship such as a vertical relationship. The result output by the analysis result output unit functions as an improvement candidate holding unit that holds in the storage device the items that the system has determined to have not been sufficiently analyzed,
10. The program according to claim 8, wherein the program is made to function as improvement candidate data presentation means for presenting improvement candidate data in accordance with an instruction of a user who maintains each dictionary.

Images