JP5703629B2 - Synonym dictionary generation device, data analysis device, data detection device, synonym dictionary generation method, and synonym dictionary generation program - Google Patents

Description

  The present invention relates to a search system technique for extracting synonyms.

  In recent years, with the development of a communication terminal device such as a personal computer or a network such as the Internet, data desired by a user is obtained based on a predetermined search key such as a keyword from a plurality of data stored in advance in a database. Search systems to detect and analysis systems to analyze data have become common. Recently, in addition to detection of document data having the keyword or analysis of data using a keyword, a system for detecting document data that is not completely identical to the keyword and that matches a synonym of the keyword. Has been devised, and it is possible to realize accurate detection of document data desired by a user and accurate analysis of data.

  Under such circumstances, in a data search system or data analysis system using synonyms, it is possible to pre-define and group synonyms for keywords for search or analysis (that is, to create a dictionary). It has become important. For example, as a conventional data search system, one that automatically extracts synonyms based on parentheses is known (for example, Patent Document 1). Specifically, this data search system performs morphological analysis on predetermined sentence data, detects an open parenthesis symbol “(” for the sentence data, and detects the symbol, The word specified by the part of speech immediately after the opening parenthesis is recognized as a synonym in the word specified by the part of speech immediately before the opening parenthesis.

Japanese Patent Laid-Open No. 5-298371

  However, in the data search system described in Patent Document 1, a synonym is detected based on a special symbol of an open parenthesis, and is synonymous if it is not defined in a target sentence by a special symbol. It cannot be detected as a word, and a synonym cannot be extracted from the relationship between ordinary sentences, so it is not very versatile. Also, in this data search system, since it cannot be detected as a synonym unless it exists in the same sentence, synonyms cannot be extracted based on other different sentences. Cannot be detected.

  The present invention has been made to solve the above-described problems, and has an object of being able to detect synonyms using words used in a plurality of different sentences and having versatility. Another object of the present invention is to provide a synonym dictionary generation device capable of defining synonyms widely, a data search device and a data analysis device using the same.

  (1) In order to solve the above-described problem, the synonym dictionary generation device of the present invention includes an acquisition unit that externally acquires a plurality of document data to which category information indicating the same category is assigned with respect to the contents of a common document. Extraction means for performing morphological analysis and syntactic analysis on the document data, and extracting an evaluation expression text indicating an evaluation expression and a text set of modified word text modified by the evaluation expression text together with the category information; Totalizing means for totalizing the number of occurrences of the modified word text for each category and for each identical evaluation expression, and other modified objects having the same evaluation expression as each modified word text for each same category A modified word set generating means for generating a modified word set with the word text, and for each of the generated modified word sets, each modified word set Based on each appearance frequency number in the ornament text, the set appearance frequency number defining means for defining the set appearance frequency number and the set appearance frequency number in the modified word set composed of the same modified word text in all categories Calculating means for adding together, and detecting a modified word set having a predetermined condition for the number of set appearance frequencies, and defining each modified word text belonging to the detected modified word set as a synonym And synonym dictionary generating means for generating a synonym dictionary.

  With this configuration, the synonym dictionary generation device of the present invention defines a set of words that are sets of modified word texts in the same evaluation expression in the same category as the synonyms using the category information and the frequency of appearance. A synonym dictionary can be generated. Generally, when the same evaluation expression is used for the contents of a common document, different modifier texts modified by the same evaluation expression are synonymous. Therefore, according to the present invention, synonyms can be defined from a plurality of different sentences and words belonging to a plurality of different documents by using the category information. Can also be defined as synonyms in unusual words, such as a specialized word in a specialized field or a specific industry. As a result, the present invention can generate a synonym dictionary that has versatility and in which synonyms are widely defined.

  (2) Further, in the synonym dictionary generation device of the present invention, the appearance frequency number defining means defines a frequency with a small number of appearance frequencies in each modified word text of each modified word set as a set appearance frequency number. It has the composition to do.

  With this configuration, the synonym dictionary generation device of the present invention enables the appearance frequency of one of the modified word texts even when the number of appearance frequencies of one of the modified word texts in the modified word set is significantly increased. Since it is possible to prevent the influence of the number from becoming large, the accuracy can be improved when synonyms are defined in comparison with other modified word sets.

  (3) Moreover, the synonym dictionary production | generation apparatus of this invention has the structure by which the said document data is formed from the text of the single sentence.

  With this configuration, the synonym dictionary generating apparatus of the present invention can generate a synonym dictionary by inputting text of a single sentence, and therefore detects synonyms from various documents and synonyms A dictionary can be generated.

  (4) Moreover, the synonym dictionary production | generation apparatus of this invention has the structure by which the said document data is the text of several sentences, and is formed from single category information.

  With this configuration, the synonym dictionary generation apparatus of the present invention can generate a synonym dictionary even if texts of a plurality of sentences are provided with a single category information. A synonym dictionary can be generated by detecting synonyms from the inside.

  (5) Moreover, the synonym dictionary production | generation apparatus of this invention has the structure in which the said acquisition means acquires the several document data which have the same subject from the outside.

  With this configuration, the synonym dictionary generation apparatus of the present invention can be used for the same incident, accident, or problem caused by questionnaires, web pages, newspaper or magazine articles or books, such as evaluation of events, stores, facilities, products or services. By using document data of documents that have the same subject, such as sentences written on the same topic or sentences categorized in the same genre, even highly specialized sentences or general-purpose Even if it is not a sentence, a synonym dictionary can be generated accurately.

  (6) In order to solve the above-described problem, the synonym dictionary generation device of the present invention includes an acquisition unit that externally acquires a plurality of document data to which category information indicating the same category is assigned with respect to the contents of a common document. Same as the extraction means for performing morphological analysis and syntactic analysis on the document data, and extracting the text set of the evaluation expression text indicating the evaluation expression and the modified word text modified by the evaluation expression text together with the category information Modified word set generation means for generating a modified word set for each modified word text and another modified word text having the same evaluation expression for each category, and a modified word set having a predetermined criterion Synonym dictionary generating means for generating a synonym dictionary by defining each set of modified word text belonging to .

  With this configuration, the synonym dictionary generation device of the present invention defines a set of words that are sets of modified word texts in the same evaluation expression in the same category as the synonyms using the category information and the frequency of appearance. A synonym dictionary can be generated. Generally, when the same evaluation expression is used for the contents of a common document, different modifier texts modified by the same evaluation expression are synonymous. Therefore, according to the present invention, synonyms can be defined from a plurality of different sentences and words belonging to a plurality of different documents by using the category information. Can also be defined as synonyms in unusual words, such as a specialized word in a specialized field or a specific industry. As a result, the present invention can generate a synonym dictionary that has versatility and in which synonyms are widely defined.

  (7) In order to solve the above-described problem, the data analysis apparatus of the present invention includes an acquisition unit that externally acquires a plurality of learning data to which category information indicating the same category is given for the contents of a common document, Extraction means for performing morphological analysis and syntax analysis on the learning data, and extracting the evaluation expression text indicating the evaluation expression and a text set of the modified word text modified by the evaluation expression text together with the category information; and the category A totaling means for counting the frequency of appearance of the modified word text for each and the same evaluation expression, and another modified word text having the same evaluation expression as each modified word text for the same category A modified word set generating means for generating a modified word set and a modified word set for each of the generated modified word sets Based on each appearance frequency number in the text, the set appearance frequency number defining means for defining the set appearance frequency number and the set appearance frequency number in the modified word set composed of the same modified word text are added up in all categories. Calculating means and detecting a modified word set having a condition in which the set appearance frequency count is predetermined, and defining each modified word text belonging to the detected modified word set as a synonym; Synonym dictionary generating means for generating a synonym dictionary, detecting means for detecting the number of appearance frequencies in the learning data as the synonym appearance frequency number for each of the defined synonyms, and the detected synonym and appearance frequency number And an output means for outputting the synonyms in association with each other.

  With this configuration, the data analysis apparatus of the present invention uses a synonym dictionary that is defined based on words belonging to different sentences and different documents, has versatility, and has a broad definition of synonyms. Thus, even if different words are synonyms, the appearance frequency can be accurately calculated as a single word without being dispersed by various words. Usually, when opinion extraction is performed from learning data or when content is analyzed, analysis is performed based on the frequency of appearance of written words. Therefore, the present invention can accurately calculate the appearance frequency of words in learning data distributed by different word notation and generates a synonym dictionary using data to be analyzed. It is possible to extract opinions accurately or to analyze learning data accurately.

  (8) In order to solve the above-described problem, the data detection apparatus of the present invention includes an acquisition unit that externally acquires a plurality of learning data to which category information indicating the same category is given for the contents of a common document, Extraction means for performing morphological analysis and syntax analysis on the learning data, and extracting the evaluation expression text indicating the evaluation expression and a text set of the modified word text modified by the evaluation expression text together with the category information; and the category A totaling means for counting the frequency of appearance of the modified word text for each and the same evaluation expression, and another modified word text having the same evaluation expression as each modified word text for the same category A modified word set generating means for generating a modified word set and a modified word set for each of the generated modified word sets Based on each appearance frequency number in the text, the set appearance frequency number defining means for defining the set appearance frequency number and the set appearance frequency number in the modified word set composed of the same modified word text are added up in all categories. Calculating means and detecting a modified word set having a condition in which the set appearance frequency count is predetermined, and defining each modified word text belonging to the detected modified word set as a synonym; Synonym dictionary generating means for generating a synonym dictionary, and corresponding document data from among a plurality of document data stored in a database based on a keyword input by a user while using the generated synonym dictionary And a presenting means for presenting the detected document data or information related to the document data. There.

  With this configuration, the data detection device of the present invention uses a synonym dictionary that is defined based on words belonging to a plurality of different sentences and different documents, has versatility, and broadly defines synonyms. Therefore, it is possible to accurately detect document data desired by a user who cannot be detected due to different notation of words.

  (9) In order to solve the above-described problem, the synonym dictionary generation method of the present invention includes an acquisition step of acquiring a plurality of document data to which category information indicating the same category is added from the outside with respect to the contents of a common document. , An extraction step of performing morphological analysis and syntax analysis on the document data, and extracting an evaluation expression text indicating an evaluation expression and a text set of modified word text modified by the evaluation expression text together with the category information; A totaling step for counting the frequency of appearance of the modified word text for each category and for each identical evaluation expression, and other modified objects having the same evaluation expression as each modified word text for the same category A modified word set generation step for generating a modified word set with the word text, and for each of the generated modified word sets, the respective modified word set Based on each appearance frequency number in the decoration text, the set appearance frequency number defining step for defining the set appearance frequency number, and the set appearance frequency number in the modified word set composed of the same modified word text in all categories Calculating step for adding, and detecting a set of modified words having a condition in which the number of appearance frequencies of the combined set is determined in advance, and defining each modified word text belonging to the detected set of modified words as a synonym And a synonym dictionary generating step for generating a synonym dictionary.

  With this configuration, the synonym dictionary generation method of the present invention defines a set of words that are sets of modified word text in the same evaluation expression in the same category as the synonym using the category information and the frequency of appearance. A synonym dictionary can be generated. Generally, when the same evaluation expression is used for the contents of a common document, different modifier texts modified by the same evaluation expression are synonymous. Therefore, according to the present invention, synonyms can be defined from a plurality of different sentences and words belonging to a plurality of different documents by using the category information. Can also be defined as synonyms in unusual words, such as a specialized word in a specialized field or a specific industry. As a result, the present invention can generate a synonym dictionary that has versatility and in which synonyms are widely defined.

  (10) In order to solve the above-described problem, the synonym dictionary generation program of the present invention obtains a plurality of document data to which the category information indicating the same category is given from the outside with respect to the content of the common document. An extracting unit that performs morphological analysis and syntax analysis on the document data, and extracts an evaluation expression text indicating the evaluation expression and a text set of the modified word text modified by the evaluation expression text together with the category information; A counting means for counting the number of appearance frequencies of the modified word text for each category and for the same evaluation expression; another qualified having the same evaluation expression as each modified word text for the same category A modified word set generating means for generating a modified word set with a word text, and for each generated modified word set, each modified word set Set appearance frequency number defining means for defining the set appearance frequency number based on each appearance frequency number in each of the modified word text, and the set appearance frequency number in the modified word set composed of the same modified word text Calculating means for summing up in all categories, and a modifiable word set having a condition in which the sum of set appearance frequency counts is predetermined, and each qualified word text belonging to the detected qualified word set A synonym dictionary generating unit that defines a synonym and generates a synonym dictionary is configured to function.

  With this configuration, the synonym dictionary generation program of the present invention uses the category information and the frequency of appearance to define as a synonym a set of words that are sets of modified word text in the same evaluation expression in the same category. A synonym dictionary can be generated. Generally, when the same evaluation expression is used for the contents of a common document, different modifier texts modified by the same evaluation expression are synonymous. Therefore, according to the present invention, synonyms can be defined from a plurality of different sentences and words belonging to a plurality of different documents by using the category information. Can also be defined as synonyms in unusual words, such as a specialized word in a specialized field or a specific industry. As a result, the present invention can generate a synonym dictionary that has versatility and in which synonyms are widely defined.

  The synonym dictionary generation device, the synonym dictionary generation method, and the synonym dictionary generation program of the present invention can define synonyms from words belonging to different sentences and different documents by using category information. As long as the category information is given, synonyms can be defined from various documents, and therefore, synonyms can be defined in words that are not common, such as words unique to a specialized field or a specific industry. Therefore, the present invention can generate a synonym dictionary having versatility and in which synonyms are widely defined.

  In addition, since the data analysis apparatus of the present invention can accurately calculate the appearance frequency of words in learning data distributed by different word notations, and generates a synonym dictionary using data to be analyzed. It is possible to extract opinions accurately from learning data, or to analyze learning data accurately.

  In addition, the data detection apparatus of the present invention is a document that uses a synonym dictionary that is defined based on words belonging to a plurality of different sentences and a plurality of different documents, has versatility, and broadly defines synonyms. Since data can be detected, it is possible to accurately detect document data desired by a user who cannot be detected due to different word expressions.

1 is a system configuration diagram showing a configuration of a first embodiment in a document analysis system according to the present invention. In 1st Embodiment, it is a figure for demonstrating the questionnaire data and category information to acquire. In 1st Embodiment, it is a figure for demonstrating the relationship between the extracted text set and category information. In 1st Embodiment, it is a figure for demonstrating a to-be-modified word set and a set appearance frequency number. In 1st Embodiment, it is a figure for demonstrating the to-be-modified word set of all the categories totaled. In 1st Embodiment, it is a figure for demonstrating the learning data by which the data analysis was carried out. It is a flowchart which shows the operation | movement of the data analysis process of 1st Embodiment. It is a system configuration figure showing the composition of a 2nd embodiment in the document detection system concerning the present invention. It is a flowchart which shows the operation | movement of the document detection process of 2nd Embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the embodiment described below, a document analysis system for analyzing a questionnaire using a customer's answer document in a questionnaire (hereinafter simply referred to as “questionnaire”) as document data (learning data), or a plurality of document data In the embodiment, the synonym dictionary generation device, the data analysis device, the data detection device, and the like of the present invention are applied to a document search system that detects corresponding document data from the above.

<First Embodiment>
First, an embodiment of a document analysis system 100 according to the present invention will be described with reference to FIGS.

[Overview of document analysis system]
First, a schematic configuration of the document analysis system 100 according to the present embodiment will be described with reference to FIG. FIG. 1 is a system configuration diagram showing the configuration of the document analysis system 100 of the present embodiment.

  The document analysis system 100 is a computer device having various functions such as a PC (Personal Computer) or a server device. In particular, the document analysis system 100 includes each piece of questionnaire data having category information indicating the same category for the contents of a common document, and the questionnaire data on the same subject such as evaluation of an event, a store, a facility, a product, or a service. While using the questionnaire data as learning data, a synonym dictionary is generated and data analysis of the learning data is performed.

  As shown in FIG. 1, the document analysis system 100 includes an input interface 110, a communication interface 120, a data storage unit 130, a display 170, a display control unit 150 that controls the display 170, and various hardware such as a ROM / RAM 160. A data processing unit 140 that executes predetermined processing and a system management control unit 180 that manages the entire system are included. Then, the document analysis system 100 obtains the evaluation expression text indicating the evaluation expression and the evaluation text for each questionnaire data acquired through the input interface 110 or the communication interface 120 or read out from the data storage unit 130. The synonym is defined based on the evaluation expression text, the category information, and the appearance frequency of the modified word text, while extracting a set of texts composed of the modified words modified by as a text set. Generally, when the same evaluation expression is used for the contents of a common document, different modifier texts modified by the same evaluation expression are synonymous. Therefore, the document analysis system 100 according to the present embodiment uses this property, and in the case where a predetermined condition is satisfied in a set of modified word text generated based on the same evaluation expression, that is, for each evaluation expression. In addition, the number of occurrences of the modified word text of each document data totaled for each category is totaled, and the qualified word belonging to the set of qualified words when the total number of appearance frequencies satisfies a predetermined criterion Each text set is defined as a synonym and a synonym dictionary is generated.

  In addition, the document analysis system 100 performs data analysis of questionnaire data while using the defined synonyms as a synonym dictionary, and extracts opinions from the questionnaire data. Specifically, the document analysis system 100 calculates the number of appearance frequencies (hereinafter, also referred to as “synonym appearance frequency number”) in the questionnaire data for the modified word text including synonyms, and the calculation is performed. The result can be presented as a result of opinion extraction as a frequent opinion of the user who executed the questionnaire.

  In this embodiment, the input interface 110, the communication interface 120, the data storage unit 130, the display 170, the display control unit 150, the ROM / RAM 160, the data processing unit 140, and the system management control unit 180 are connected to each other via the bus B. Yes.

[System configuration]
Next, the details of the system configuration in the document analysis system 100 of this embodiment will be described with reference to FIG.

  The input interface 110 is an input device such as a touch sensor provided on a keyboard, a mouse, or a display 170 and used for user operation input or data input of questionnaire data. Specifically, the input interface 110 is used when inputting questionnaire data to be analyzed and performing operations for executing each process described later during execution of the analysis process on the questionnaire data. .

  The communication interface 120 is connected to the network N, constructs a communication line with other communication devices such as the server device 10 or the database 300, and exchanges various data. In particular, the communication interface 120 is configured to exchange necessary data during execution of a questionnaire data analysis process such as acquisition of questionnaire data in conjunction with the data processing unit 140.

  The data storage unit 130 is formed so that a plurality of questionnaire data can be stored in advance in association with category information. In particular, the data storage unit 130 can store questionnaire data acquired via the input interface 110 or the communication interface 120. The data storage unit 130 stores synonyms defined by analyzing the questionnaire data as synonym dictionary data.

  The display 170 is a display device formed by, for example, a liquid crystal element, an EL (Electro Luminescence) element, or a CRT, and displays a predetermined image based on display data generated by the display control unit 150. . In particular, the display 170 displays various images such as operation input instructions and data analysis results in conjunction with the input interface 110 during execution of the questionnaire data analysis process. Note that the display 170 device may be incorporated in a single casing together with other hardware, or may be provided separately from other hardware. Further, for example, the display 170 of the present embodiment constitutes an output unit of the present invention.

  The display control unit 150 generates drawing data necessary for drawing a predetermined image on the display 170 under the control of the system management control unit 180 or under the control of the data processing unit 140. The rendered drawing data is output to the display 170.

  Various control programs for functioning as the document analysis system 100 are recorded in the ROM / RAM 160. The ROM / RAM 160 is used as a work area when various processes such as temporary storage of the acquired questionnaire data are executed during the execution of the analysis process of the questionnaire data.

The data processing unit 140 cooperates with the above-described hardware, and analyzes the questionnaire data, that is, a program for analyzing the acquired questionnaire data and defining the synonyms (hereinafter referred to as the questionnaire) , "Questionnaire totaling program"). In particular, as a process executed by the questionnaire totalization program,
(1) An acquisition process for acquiring a plurality of questionnaire data (that is, document data) that includes category information and is composed of single text data in text format (hereinafter also referred to as “text text”);
(2) From the acquired questionnaire data, together with the category information, an evaluation expression text indicating the evaluation expression and a text set of the modified word text modified by the evaluation expression text are extracted, and the appearance frequency of each modified word text for each category Extraction process to calculate the number,
(4) For each same category, a modified word set is generated for each modified word text and another modified word text having the same evaluation expression, and the appearance frequency number as the modified set is set as the set appearance frequency number. Set appearance frequency count definition processing to define,
(5) A modified word set having the set appearance frequency number in the modified word set composed of the same modified word text added up in all categories, and the combined set appearance frequency number having a predetermined condition. And generating a synonym dictionary by defining each modified word text belonging to the detected modified word set as a synonym, and (6) obtaining using the generated synonym dictionary Data analysis of multiple questionnaire data,
Each process is executed.

  Specifically, as shown in FIG. 1, the data processing unit 140 executes a questionnaire totalization program, as shown in FIG. 1, a data acquisition processing unit 141, an extraction processing unit 142, an appearance frequency number defining unit 143, and a synonym defining unit 144. And the data analysis unit 145 is constructed.

  Details of the configuration and operation of the data processing unit 140 of this embodiment will be described later. Further, for example, the data acquisition processing unit 141 of the present embodiment constitutes an acquisition unit of the present invention, and the extraction processing unit 142 constitutes an extraction unit and a tabulation unit of the present invention. Further, for example, the appearance frequency number defining unit 143 of the present embodiment constitutes the modified word set generating unit and the appearance frequency number defining unit of the present invention, and the synonym defining unit 144 is the calculating unit and synonym of the present invention. While constituting the dictionary generation means, the data analysis unit 145 constitutes the detection means of the present invention.

  The system management control unit 180 is mainly composed of a central processing unit (CPU) and includes various input / output ports such as a key input port and a display control port. It is designed to control overall functions for executing programs.

[Data processing section]
Next, the configuration and operation of each unit in the data processing unit 140 according to the present embodiment will be described with reference to FIGS.

  2 is a diagram for explaining questionnaire data and category information to be acquired in this embodiment, and FIG. 3 is for explaining the relationship between the extracted text set and category information in this embodiment. FIG. Moreover, FIG. 4 is a figure for demonstrating a to-be-modified word set and the set appearance frequency number in this embodiment, and FIG. 5 demonstrates the to-be-modified word set of all the categories totaled in this embodiment. It is a figure for doing. Furthermore, FIG. 6 is a diagram for explaining learning data subjected to data analysis in the present embodiment.

  The data acquisition processing unit 141 acquires each questionnaire data (that is, sentence text) based on an interactive application program in cooperation with the input interface 110, the display 170, and the display control unit 150. In particular, the data acquisition processing unit 141 acquires each questionnaire data together with category flag type information (hereinafter referred to as “category flag”) which is category information input through the input interface 110. .

  For example, as shown in FIG. 2, the data acquisition processing unit 141 uses a sentence text “Restaurant employee was kind”, a category flag “3 (about a clerk)”, a sentence text “high admission” and a category flag. “2 (about the price)”, sentence text “the clerk is kind” and category flag “3 (about the clerk)”, sentence text “flower is beautiful” and category flag “4 (about the garden)” and sentence text “price is a little bit” Survey data such as “high” and category flag “2 (about price)” and category information are acquired. In addition, each acquired questionnaire data may be used for a questionnaire totalization program directly after input with category information, or may be temporarily stored in the data storage unit 130.

  Note that the data acquisition processing unit 141 may acquire each questionnaire data having category information from the server device 10 or the database 300 via the communication interface 120 instead of the input from the input interface 110. In addition, the data acquisition processing unit 141 may add the category information to the questionnaire data to which the category information acquired by the communication interface 120 is not added by input from the input interface 110.

  On the other hand, the data acquisition processing unit 141 may acquire each questionnaire data by reading each questionnaire data stored in the data storage unit 130 instead of input from the input interface 110 and acquisition by the communication interface 120. . The data acquisition processing unit 141 may acquire each questionnaire data by reading from a removable recording medium (not shown) such as a CD, a DVD, or a flash memory instead of the data storage unit 130.

  The extraction processing unit 142 performs a morphological analysis and a syntactic analysis on each acquired questionnaire data to show an evaluation expression text indicating an evaluation expression and a text set of the modified word text modified by the evaluation expression text together with category information. The number of appearances of the modified word text is tabulated for each category indicated by the category information and for the same evaluation expression.

  Specifically, the extraction processing unit 142 may express the value of the object of quality or quantity including “kind”, “high”, “clean”, “easy” other nouns, adjectives, adjective verbs, or emotions. Evaluation expression text list data (hereinafter also referred to as “evaluation expression dictionary data”). Then, the extraction processing unit 142 performs morphological analysis and syntax analysis on the input sentence text based on the evaluation expression dictionary data, and matches the evaluation expression text in the list data from the sentence text ( Text). Further, the extraction processing unit 142 detects the phrase that the matched text is on the sentence text, that is, the phrase that the text modifies on the sentence text (that is, the modified word text), and the evaluation expression text and the modified text A combination of word texts (text set) is extracted. For each extracted text set, the extraction processing unit 142 extracts the appearance frequency number of the modified word text belonging to each text set for each category and totals the appearance frequency numbers.

  For example, when the questionnaire data and category information shown in FIG. 2 are acquired, the extraction processing unit 142, as shown in FIG. 3A, the text sets “kind” and “ “Employee”, “High” and “Admission”, “Friendly” and “Clerk” are extracted, and category information in each text set is extracted. Then, as shown in FIG. 3 (b), the extraction processing unit 142 sets “high” for each category flag of category 2 (for price), category 3 (for clerk), and category 4 (for garden). For each evaluation expression text of "Cheap", "Friendly", "Unfriendly", "Easy" and "Pretty", we will extract the modifier text such as "Price", "Admission", "Food", etc. The appearance frequency of each modifier text for each evaluation expression text is extracted, and the number of appearance frequencies is totaled.

  It should be noted that the extraction processing unit 142 extracts, as evaluation expression text, idioms composed of a plurality of morphemes, for example, “reachable to an ugly place” and “remaining eyes” as evaluation expression text. ing.

  The appearance frequency number definition unit 143 is configured of two modified word texts in association with other modified word texts belonging to other text sets having the same evaluation expression as the modified word texts for each same category. The modified word set is generated, and the appearance frequency number as the modified set is defined as the set appearance frequency number. In particular, the appearance frequency number definition unit 143 determines the common appearance frequency number of each modified word text belonging to the modified word set, that is, the smaller appearance frequency number of each modified word text belonging to the modified word set. This is defined as the number of set appearance frequencies.

  For example, as shown in FIG. 3, the appearance frequency number definition unit 143 extracts the text sets of “price”, “entrance fee”, and “food” for the same evaluation expression text “high” for category 2, When the appearance frequency numbers are respectively counted as “20”, “7”, and “2”, as shown in FIG. 4, “price” and “entrance fee”, “price” and “food”, and “ A combination set of modifiers for each of “entrance fee” and “food” is generated. Then, the appearance frequency count definition unit 143 displays the appearance frequencies “20” and “7”, “20”, and “20” of the modified word text (modified word text A and B shown in FIG. 4) belonging to each modified word set. “2”, “7”, and “2” are compared, and “7”, “2”, and “2”, which have a small number of appearance frequencies, are defined as the number of appearance frequencies of each modified word set. . That is, as shown in FIG. 4, the appearance frequency number definition unit 143 generates a modified word set for each category and defines it as the set appearance frequency number.

  The synonym definition unit 144 adds the set appearance frequency numbers in the modified word set composed of the modified word texts for the same evaluation expression text in all categories, and the set appearance frequency number is a predetermined threshold value. A modified word set having predetermined conditions such as the top three having the highest number of appearance frequencies or the set is detected. And the synonym definition part 144 defines each to-be-modified word text which belongs to the said to-be-modified word set as a synonym, and produces | generates the list | wrist of the defined synonym as a synonym dictionary. ing.

  For example, as shown in FIG. 5, the synonym definition unit 144 includes “price” and “fee”, “employee” and “clerk”, “attraction” and “vehicle”, and the like generated for each category. A set of appearance frequencies of the same set of modified words in which each of the modified word texts A and B in the modifier set is the same, and a threshold set having a frequency of appearance frequency “100” or more as a predetermined condition; The modified word texts A and B of “price” and “fee”, “employee” and “store clerk”, “attraction” and “vehicle” are defined as synonyms.

  The data analysis unit 145 performs questionnaire analysis such as frequency counting of texts in the questionnaire (for example, modified word text) based on a plurality of questionnaire data acquired using the synonym dictionary generated by the synonym definition unit 144. Processing is to be executed.

  Specifically, when frequent opinions are extracted as questionnaire analysis processing, the data analysis unit 145 uses the synonym dictionary shown in FIG. 6A to qualify questionnaire data used as learning data. The frequency of word text is extracted for each synonym and presented on the display 170 or the like. For example, in the case where a synonym dictionary is not used and frequent opinions are extracted, as shown in FIG. 6B, “clerk”, “employee”, “fee”, and “entrance fee” Appearance frequencies are calculated separately as different modifier texts. However, the data analysis unit 145 according to the present embodiment executes the above-described processes, thereby defining the qualifier texts of “fee” and “entrance fee” and “clerk” and “employee” as synonyms. Therefore, as shown in FIG. 6C, the data analysis unit 145 adds the appearance frequency numbers of “fee” and “clerk” to the appearance frequency numbers of the synonyms “admission fee” and “employee”. To be presented.

  As a result, the data analysis unit 145 can accurately extract the modified word text as frequent opinion extraction. In addition, the questionnaire analysis process calculates the frequency of occurrence for each modified word text, and sums up the frequency of occurrence of each modified word text calculated based on the synonym dictionary for each synonym and re-aggregates You may make it do. That is, in this case, in the questionnaire analysis process, the calculation in FIG. 6B may be executed first, and then the recalculation in FIG. 6C may be executed.

[Data analysis processing]
Next, the operation of the data analysis process in this embodiment will be described with reference to FIG. FIG. 7 is a flowchart showing the operation of the data analysis process in this embodiment.

  This operation will be described for a case where a plurality of questionnaire data is acquired while providing category information using the input interface 110. In addition, it is assumed that a data analysis program for acquiring a plurality of questionnaire data has already been activated.

  First, when the data acquisition processing unit 141 detects a user instruction to start inputting questionnaire data via the input interface 110 (step S101: YES), the data acquisition processing unit 141 is linked with the input interface 110 and the display control unit 150, While performing various displays on the display 170, the questionnaire data input by the user and the category information corresponding to each questionnaire data are acquired (step S102). At this time, the data acquisition processing unit 141 executes an interactive application program to acquire questionnaire data and its category information, and the data acquisition processing unit 141 temporarily stores it in the data storage unit 130.

  Next, the extraction processing unit 142 performs morphological analysis and syntax analysis on each acquired questionnaire data (step S103), and the evaluation expression text indicating the evaluation expression and the modified word text to be modified by the evaluation expression text, Are extracted together with category information (step S104).

  Next, the extraction processing unit 142 counts the appearance frequency number of the modified word text for each category indicated by the category information and for each identical evaluation expression (step S105), and the appearance frequency number definition unit 143 Then, a modified word set is generated for each same category, and the appearance frequency number as the modified set is defined as the set appearance frequency number (step S106).

  Next, the synonym definition unit 144 adds the set appearance frequency numbers in the modified word set composed of the same modified word text in all categories (step S107), and the set appearance frequency number is predetermined. A modified word set having a condition equal to or greater than a threshold value is detected, and each modified word text belonging to the detected modified word set is defined as a synonym and a synonym dictionary is generated (step S108).

  Next, the data analysis unit 145 executes a questionnaire analysis process that counts the frequency numbers of texts (for example, modified word texts) in a plurality of questionnaire data acquired using the synonym dictionary generated by the synonym definition unit 144. (Step S109).

  Finally, the data analysis unit 145 presents the result of the questionnaire analysis process on the display 170 (step S110) and ends the operation.

  In addition, although this operation | movement demonstrated the case where a some questionnaire data was acquired, providing category information using the input interface 110, when acquiring from the server apparatus 10 or the database 300 via the communication interface 120, it is. The data acquisition processing unit 141 acquires a plurality of questionnaire data while giving category information in conjunction with the communication interface 120.

[Function and effect]
As described above, the document analysis system 100 according to the present embodiment can define synonyms from words belonging to a plurality of different sentences and different documents by using category information. Since synonyms can be defined from the document, a synonym dictionary can be generated that also defines synonyms in unusual words such as words specific to a specialized field or a specific industry. Therefore, the document analysis system 100 according to the present embodiment is versatile and can perform data analysis using a synonym dictionary in which synonyms are widely defined. The document analysis system 100 according to the present embodiment can accurately calculate the appearance frequency of words in learning data that has been distributed due to different notation of words in the document, so that opinions can be accurately extracted from the learning data. Or learning data can be analyzed accurately.

  In addition, the document analysis system 100 according to the present embodiment defines a frequency with a small appearance frequency number in each modified word text of each modified word set as a set appearance frequency number. Even if the number of occurrences of the modified word text is significantly increased, it is possible to prevent the influence of the appearance frequency number of one of the modified word texts from increasing, so that In comparison, data analysis can be performed using a synonym dictionary that can accurately define synonyms.

  In other words, if the frequency of occurrence of one of the modified word texts in the modified word set is significantly large, defining the large frequency as the set frequency of occurrence, the frequency of occurrence of each modified word set is the largest frequency of occurrence Will be unified. For example, in the case of the example in FIG. 3, the number of appearance frequencies in the combination of each modified word set for the evaluation expression text “high” is all “20”. Therefore, if the appearance frequency numbers in all categories are added together, it becomes difficult to produce a difference in the appearance frequency numbers in each modified word set, and many combinations of the modified word sets are defined as synonyms. Therefore, such a situation can be prevented by defining a frequency with a small number of appearance frequencies in each modified word text of each modified word set as a set appearance frequency number.

  In addition, the document analysis system 100 according to the present embodiment can generate a synonym dictionary by inputting a single sentence text. Therefore, synonyms generated by detecting synonyms from various documents. Data analysis can be performed using a word dictionary.

[Modification]
Next, a modified example of the document analysis system 100 of this embodiment will be described.

  The appearance frequency number defining unit 143 according to the present embodiment is configured to define the appearance frequency number of each of the modified word texts belonging to the modified word set as the set appearance frequency number. It is only necessary that the set appearance frequency number can be determined by a certain rule such as using a predetermined function based on the average of both and the appearance frequency number of both or one.

  Further, the data analysis unit 145 of the present embodiment is configured to present the result of the questionnaire analysis process on the display 170, but may be printed and output by a printer or the like, or the data storage unit 130 or the recording medium It may be output as data.

  The data processing unit 140 of the present embodiment uses questionnaire data formed from a single sentence, but is a text of a plurality of sentences and formed from a single category information. Also good. In this case, the extraction processing unit 142 performs morphological analysis and syntax analysis according to each sentence. Therefore, if a single category information is given, a synonym dictionary can be generated even for texts of multiple sentences, so synonyms can be detected from various documents by detecting synonyms. It can be generated.

  In this embodiment, questionnaire data having the same subject is used as document data. However, the same topic such as the same incident, accident, or problem is described by an article or book of a Web page, newspaper or magazine. So that a synonym dictionary can be generated by adding category information to texts that have the same subject, such as texts that have been categorized in the same genre It has become.

Second Embodiment
First, an embodiment of the document detection system 200 according to the present invention will be described with reference to FIGS. 8 and 9.

  In the present embodiment, instead of performing the frequency aggregation of the text on the questionnaire data used as the learning data in the first embodiment, a document having a specific text (for example, a modified word text) is detected. There are features and other configurations are the same as those of the first embodiment. In the present embodiment, the same members as those in the first embodiment are denoted by the same reference numerals, and the description thereof is omitted.

[Overview of document detection system]
First, a schematic configuration of the document detection system 200 according to the present embodiment will be described with reference to FIG. FIG. 8 is a system configuration diagram showing the configuration of the document detection system 200 of this embodiment.

  As shown in FIG. 8, the document analysis system 100 includes an input interface 110, a communication interface 120, a data storage unit 130, a display 170, a display control unit 150 for controlling the display 170, and various hardware such as a ROM / RAM 160. A data processing unit 240 that executes predetermined processing and a system management control unit 180 that manages the entire system are included. In particular, as shown in FIG. 8, the data processing unit 240 of the present embodiment executes a program for realizing document detection processing (hereinafter referred to as “document detection program”) instead of the questionnaire totalization program. The data acquisition processing unit 141, the extraction processing unit 142, the appearance frequency number definition unit 143, the synonym definition unit 144, and the document data detection unit 241 are constructed.

  Note that the input interface 110, the communication interface 120, the data storage unit 130, the display 170, the display control unit 150, the ROM / RAM 160, the data processing unit 240, and the system management control unit 180 in this embodiment are connected to each other by a bus B. Yes. Further, for example, the display 170 of the present embodiment constitutes the presenting means of the present invention.

[Document data detector]
The document data detection unit 241 detects corresponding document data from the document data stored in advance in the data storage unit 130 or the database 300 using the synonym dictionary generated by the synonym definition unit 144 and displays it on the display 170. The detection result is displayed.

  Specifically, the document data detection unit 241 compares each text included in each document data with the text listed in the synonym dictionary including synonyms, and detects document data having matching text. It is like that. The document data detection unit 241 displays information on the document data such as the content of the detected document data, the data name, or the recorded address on the display 170. Note that, for example, the document data detection unit 241 of the present embodiment constitutes the detection means of the present invention.

[Document detection processing]
Next, the operation of the document detection process in this embodiment will be described with reference to FIG. FIG. 9 is a flowchart showing the operation of the document detection process in this embodiment.

  This operation will be described for a case where a plurality of questionnaire data is acquired while providing category information using the input interface 110. Also, it is assumed that a document detection processing program for acquiring a plurality of questionnaire data has already been started.

  First, when the data acquisition processing unit 141 detects a user instruction to start inputting questionnaire data via the input interface 110 (step S201: YES), the data acquisition processing unit 141 works in conjunction with the input interface 110 and the display control unit 150, While performing various displays on the display 170, the questionnaire data input by the user and the category information corresponding to each questionnaire data are acquired (step S202). At this time, the data acquisition processing unit 141 temporarily stores the data in the data storage unit 130.

  Next, the extraction processing unit 142 performs morphological analysis and syntax analysis on each acquired questionnaire data (step S203), and the evaluation expression text indicating the evaluation expression, the modified word text to be modified by the evaluation expression text, Are extracted together with category information (step S204).

  Next, the extraction processing unit 142 counts the appearance frequency number of the modified word text for each category indicated by the category information and for each identical evaluation expression (step S205), and the appearance frequency number definition unit 143 For each same category, a modified word set is generated, and the appearance frequency number as the modified set is defined as the set appearance frequency number (step S206).

  Next, the synonym definition unit 144 adds the set appearance frequency numbers in the modified word set composed of the same modified word text in all categories (step S207), and the set appearance frequency number is predetermined. A modified word set having a condition equal to or greater than a threshold value is detected, and each modified word text belonging to the detected modified word set is defined as a synonym and a synonym dictionary is generated (step S208).

  Next, the document data detection unit 241 uses the synonym dictionary generated by the synonym definition unit 144 to list each text in the plurality of document data stored in the data storage unit 130 in advance in the synonym dictionary. The existing texts are compared, and the document data having the matching text is detected (step S209).

  Finally, the document data detection unit 241 displays information on the document data such as the content of the detected document data, the data name, or the recorded address on the display 170 (step S210), and ends the operation.

  In addition, although this operation | movement demonstrated the case where a some questionnaire data was acquired, providing category information using the input interface 110, when acquiring from the server apparatus 10 or the database 300 via the communication interface 120, it is. The data acquisition processing unit 141 acquires a plurality of questionnaire data while giving category information in conjunction with the communication interface 120.

[Function and effect]
As described above, the document detection system 200 according to the present embodiment can define synonyms from words belonging to a plurality of different sentences and a plurality of different documents by using category information. Since synonyms can be defined from the document, a synonym dictionary can be generated that also defines synonyms in unusual words such as words specific to a specialized field or a specific industry. Therefore, the document detection system 200 according to the present embodiment is versatile and can perform data analysis using a synonym dictionary in which synonyms are widely defined. The document analysis system 100 according to the present embodiment can accurately calculate the appearance frequency of words in learning data that has been distributed due to different notation of words in the document, so that opinions can be accurately extracted from the learning data. Or learning data can be analyzed accurately.

  In addition, since the document detection system 200 of the present embodiment defines a frequency with a small number of appearance frequencies in each modified word text of each modified word set as a set appearance frequency number, Even if the number of occurrences of the modified word text is significantly increased, it is possible to prevent the influence of the appearance frequency number of one of the modified word texts from increasing, so the other modified word set , The document data can be detected using a synonym dictionary that can accurately define synonyms.

  In addition, the document detection system 200 according to the present embodiment can generate a synonym dictionary by inputting a single sentence text. Therefore, synonyms generated by detecting synonyms from various documents. Document data can be detected using a word dictionary.

[Modification]
Next, a modified example of the document analysis system 100 of this embodiment will be described.

  The appearance frequency number defining unit 143 according to the present embodiment is configured to define the appearance frequency number of each of the modified word texts belonging to the modified word set as the set appearance frequency number. It is only necessary that the set appearance frequency number can be determined by a certain rule such as using a predetermined function based on the average of both and the appearance frequency number of both or one.

  Further, the data analysis unit 145 of the present embodiment is configured to present the result of the questionnaire analysis process on the display 170, but may be printed and output by a printer or the like, or the data storage unit 130 or the recording medium It may be output as data.

  In addition, the data processing unit 240 of the present embodiment uses questionnaire data formed from a single sentence, but is a text of a plurality of sentences and formed from a single category information. Also good. In this case, the extraction processing unit 142 performs morphological analysis and syntax analysis according to each sentence. Therefore, if a single category information is given, a synonym dictionary can be generated even for texts of multiple sentences, so synonyms can be detected from various documents by detecting synonyms. It can be generated.

  In this embodiment, questionnaire data having the same subject is used as document data. However, the same topic such as the same incident, accident, or problem is described by an article or book of a Web page, newspaper or magazine. So that a synonym dictionary can be generated by adding category information to texts that have the same subject, such as texts that have been categorized in the same genre It has become.

  The document analysis system 100 and the document detection system 200 of the present invention can be applied to all fields of document data analysis and document data detection such as questionnaire analysis and document detection.

B ... Bus 10 ... Server device 100 ... Document analysis system 110 ... Input interface 120 ... Communication interface 130 ... Data storage unit 140, 240 ... Data processing unit 141 ... Data acquisition processing unit 142 ... Extraction processing unit 143 ... Appearance frequency number definition unit 144… synonym definition unit 145… data analysis unit 150… display control unit 160… ROM / RAM
170 ... Display 180 ... System management control unit 200 ... Document detection system 241 ... Document data detection unit 300 ... Database

Claims (8)

With respect to the content of the common document, an acquisition unit that acquires a plurality of document data to which category information indicating the same category is attached from outside,
Extraction means for performing morphological analysis and syntax analysis on the document data, and extracting a text set of an evaluation expression text indicating an evaluation expression and a modified word text modified by the evaluation expression text together with the category information;
Aggregating means for aggregating the number of appearance frequencies of the modified word text for each category and for each identical evaluation expression;
A modified word set generating means for generating a modified word set with each modified word text and another modified word text having the same evaluation expression for each category,
For each generated modified word set, the number of appearance frequencies in each modified word text of each modified word set is low based on the number of appearance frequencies in each modified word text of each modified word set A set appearance frequency number defining means for defining the frequency as a set appearance frequency number;
A calculation means for adding up the number of set appearance frequencies in the modified word set composed of the same modified word text in all categories;
Detecting a modified word set having a condition in which the set appearance frequency number is predetermined, and defining each modified word text belonging to the detected modified word set as a synonym, A synonym dictionary generating means for generating;
A synonym dictionary generation device comprising: In the synonym dictionary production | generation apparatus of Claim 1,
The synonym dictionary production | generation apparatus with which the said document data is formed from the text of the single sentence. In the synonym dictionary production | generation apparatus of Claim 1 or 2,
A synonym dictionary generating apparatus, wherein the document data is a text of a plurality of sentences and formed from a single category information. In the synonym dictionary production | generation apparatus as described in any one of Claims 1 thru | or 3,
The synonym dictionary generation device, wherein the acquisition unit acquires a plurality of document data having the same subject from outside. With respect to the content of the common document, an acquisition means for acquiring a plurality of learning data to which category information indicating the same category is given from outside,
Extraction means for performing morphological analysis and syntax analysis on the learning data, and extracting a text set of an evaluation expression text indicating an evaluation expression and a modified word text modified by the evaluation expression text together with the category information;
Aggregating means for aggregating the number of appearance frequencies of the modified word text for each category and for each identical evaluation expression;
A modified word set generating means for generating a modified word set with each modified word text and another modified word text having the same evaluation expression for each category,
For each generated modified word set, the number of appearance frequencies in each modified word text of each modified word set is low based on the number of appearance frequencies in each modified word text of each modified word set A set appearance frequency number defining means for defining the frequency as a set appearance frequency number;
A calculation means for adding up the number of set appearance frequencies in the modified word set composed of the same modified word text in all categories;
Detecting a modified word set having a condition in which the set appearance frequency number is predetermined, and defining each modified word text belonging to the detected modified word set as a synonym, A synonym dictionary generating means for generating;
Detecting means for detecting an occurrence frequency number in the learning data as a synonym appearance frequency number for each of the defined synonyms;
Output means for associating and outputting the detected synonyms and the number of appearance frequencies and synonyms;
A data analysis apparatus comprising: With respect to the content of the common document, an acquisition means for acquiring a plurality of learning data to which category information indicating the same category is given from outside,
Extraction means for performing morphological analysis and syntax analysis on the learning data, and extracting a text set of an evaluation expression text indicating an evaluation expression and a modified word text modified by the evaluation expression text together with the category information;
Aggregating means for aggregating the number of appearance frequencies of the modified word text for each category and for each identical evaluation expression;
A modified word set generating means for generating a modified word set with each modified word text and another modified word text having the same evaluation expression for each category,
For each generated modified word set, the number of appearance frequencies in each modified word text of each modified word set is low based on the number of appearance frequencies in each modified word text of each modified word set A set appearance frequency number defining means for defining the frequency as a set appearance frequency number;
A calculation means for adding up the number of set appearance frequencies in the modified word set composed of the same modified word text in all categories;
Detecting a modified word set having a condition in which the set appearance frequency number is predetermined, and defining each modified word text belonging to the detected modified word set as a synonym, A synonym dictionary generating means for generating;
Using the synonym dictionary generated by the synonym dictionary generating means, comparing each text included in a plurality of document data stored in advance with each text listed in the synonym dictionary, and matching text Detecting means for detecting document data having
Presenting means for presenting the detected document data or information relating to the document data;
A data detection apparatus comprising: A synonym dictionary generation method for generating a synonym dictionary using a programmed computer,
The steps executed by the computer include
With respect to the content of the common document, an acquisition step of acquiring a plurality of document data to which category information indicating the same category is given from outside,
An extraction step of performing morphological analysis and syntax analysis on the document data, and extracting an evaluation expression text indicating an evaluation expression and a text set of a modified word text modified by the evaluation expression text together with the category information;
For each category and for each same evaluation expression, a counting step for counting the number of appearance frequencies of the modified word text;
For each same category, a modified word set generation step for generating a modified word set with each modified word text and other modified word text having the same evaluation expression;
For each generated modified word set, the number of appearance frequencies in each modified word text of each modified word set is low based on the number of appearance frequencies in each modified word text of each modified word set A set appearance frequency number defining step for defining the frequency as a set appearance frequency number;
A calculation step of summing up the number of set appearance frequencies in the modified word set composed of the same modified word text in all categories;
Detecting a modified word set having a condition in which the set appearance frequency number is predetermined, and defining each modified word text belonging to the detected modified word set as a synonym, A synonym dictionary generation step to generate;
The synonym dictionary generation method characterized by including . Computer
An acquisition means for acquiring a plurality of document data to which category information indicating the same category is given from the outside with respect to the content of a common document,
Extraction means for performing morphological analysis and syntax analysis on the document data, and extracting together with the category information an evaluation expression text indicating an evaluation expression and a text set of modified word text modified by the evaluation expression text;
A counting means for counting the frequency of appearance of the modified word text for each category and for each same evaluation expression;
A modified word set generating means for generating a modified word set with each modified word text and another modified word text having the same evaluation expression for each category,
For each generated modified word set, the number of appearance frequencies in each modified word text of each modified word set is low based on the number of appearance frequencies in each modified word text of each modified word set Set appearance frequency number defining means for defining the frequency as a set appearance frequency number;
A calculating means for adding up the set appearance frequency numbers in the modified word set composed of the same modified word text in all categories, and a modified word set having a condition in which the combined set appearance frequency number is predetermined. Synonym dictionary generating means for detecting and defining each modified word text belonging to the detected modified word set as a synonym and generating a synonym dictionary;
A synonym dictionary generation program characterized by functioning as

Images