JP3960530B2 - Text mining program, method and apparatus - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a text mining program, method, and apparatus for supporting a process of analyzing accumulated text data, grasping characteristics and trends, and discovering unknown information.
[0002]
[Prior art]
Conventionally, document classification methods, important word extraction methods, extracted word classification methods, and between extracted words have been used as techniques for analyzing accumulated text data to understand features and trends and to discover unknown information. Many text mining techniques have been proposed, such as related display methods. Among them, as a text mining visualization technique, a method of displaying an associative relationship between words in a network format as shown in FIG. 1 has been proposed. In FIG. 1, each word surrounded by a rectangle indicates a keyword extracted from the text to be analyzed, and a numerical value attached to a path connecting the keywords indicates a relationship between the keywords. In addition, “Visual Text Mining” in Journal of the Japanese Society for Artificial Intelligence Vol. 16 No. 2 (March 2001) proposes word maps, anchor maps, and skeleton maps as text mining visualization techniques.
[0003]
These are display methods for grasping a tendency by displaying a keyword that is directly related to a keyword of interest mainly on the network. Furthermore, Japanese Patent Laid-Open No. 2001-117935 and Special Table 2001-513242 have proposed a method in which when a keyword displayed on a network is clicked, a keyword related to the clicked keyword is expanded to show an indirect relationship. . However, to see indirect associations, the user had to specify keywords.
[0004]
[Problems to be solved by the invention]
In the above prior art, keywords that are directly related to a specified keyword or a plurality of keywords are displayed in a network format or a list format, so it is possible to grasp a direct relationship or a direct connection, but indirectly I couldn't figure out a typical connection. In other words, you can see the direct connection between words, but you can't see what words are between words. Therefore, there is a problem that the user can see only associative relations that can be guessed to some extent. Even when a keyword is selected and its related words are displayed gradually, there is a problem that the user has to determine and operate the search direction, and only a limited route is displayed. It was.
[0005]
The object of the present invention is to present information on what words the two keywords are connected to by displaying a plurality of related word paths from the first keyword to the second keyword in advance. By displaying the path of the related word, unknown information that has not been noticed until now is presented to the user.
[0006]
[Means for Solving the Problems]
In order to achieve the above object, the present invention is characterized by searching for a route of related words connecting two designated keywords and displaying the searched route. For example, within a range where the distance from the designated first keyword is within a predetermined value, a related word related to the first keyword and a related word connected to the first keyword via any related word are searched. A related word list of the first keyword is created from the search result, and a related word list of the second keyword is similarly created for the designated second keyword. From the related word list, both related word lists are created. A common related word that appears is searched, a route from the first keyword to the second keyword is obtained via the common related word, and the obtained route is displayed.
[0007]
Specifically, a related word dictionary creating unit that creates a related word dictionary from a document database and a related word route searching unit that obtains a route between keywords from the related word dictionary are provided. This makes it possible to create route information between the two specified keywords. In addition to the keyword specification area, an area for specifying the distance (number of related words) is provided on the input screen. As a result, it is possible to search for the path of the related word using the distance (number of related words) from the first keyword to the second keyword as a threshold value. Similarly, an input area for specifying the strength of association (relationship) between keywords is provided. As a result, it is possible to search for the path of the related word using the strength of the relation (relevance) as a threshold. As described above, the search processing is executed with the two specified keywords, distance, and relevance as inputs, and route information is created.
[0008]
In addition, as an option to display the created route information, specify the display color of the displayed route and the display color of the keyword according to the area to specify the order of the displayed route, the degree of association between keywords, and the appearance frequency of the keyword An area to be used is provided. As a method for designating the order of the routes to be displayed, there are provided a designation method by specifying the shortest route order or the longest route order and the ascending or descending order of the average value of the related strength of the route. When the shortest route order is specified, the route lengths are displayed in ascending order. When the longest route order is designated, the route lengths are displayed in order from the longest. When the ascending order of the average value of the relationship strength of the route is designated, the average value of the relationship strength between the keywords that are the elements of the route is displayed in ascending order. When the descending order of the average value of the relation strength of the route is designated, the average value of the relation strength between the keywords as the elements of the route is displayed in descending order. Further, regarding the designation of the display order, it is possible to designate whether to give priority to the route or to give priority to the average value of relevance.
[0009]
Furthermore, an option for designating the display color of the route depending on the strength of the relationship between keywords and an option for designating the display color based on the appearance frequency of the keyword may be provided. By displaying the strength of the relationship between keywords in different colors, the strength of the relationship can be grasped. In addition, by displaying the keyword frequency information in different colors, it is possible to simultaneously grasp the information of the keyword itself, that is, information about whether it is a low-frequency word or a high-frequency word. As described above, by providing the display option, the created route information can be displayed in a plurality of patterns.
[0010]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[0011]
FIG. 2 shows a configuration of a text mining apparatus according to an embodiment of the present invention. The apparatus includes a processing device 10, an input device 60, and a display device 70. The processing device 10 performs processing according to the information input from the input device 60 and displays the result on the input / output screen 80 of the display device 70. The processing device 10 includes a related word dictionary creation unit 30 that generates a related word dictionary 40 from the document database 20 in advance, and a related word path search unit 50 that searches and displays a path connecting two designated keywords. The related word dictionary creation unit 30 includes a word extraction unit 31 and a related word extraction unit 32. The related word path search unit 50 is output from the related word list creation unit 51, the related word list 52 output from the related word list creation unit 51, the related word path creation unit 53, and the related word path creation unit 53. A related word path list 54 and a related word path display unit 55 for displaying the related word path list 54 are provided.
[0012]
FIG. 3 is a flowchart showing a process until the related word dictionary creation unit 30 in FIG. 2 creates the related word dictionary 40 from the document database 20. The word extraction unit 31 reads document data as shown in FIG. 4 from the document database 20 (step 101), cuts out words as shown in FIG. 5 (step 102), and creates a word table as shown in FIG. It is created and registered in the related word dictionary 40 (step 103). As a word extraction method, there are a method of cutting out a word by referring to dictionary data, a method of cutting out a character type such as kanji or hiragana in a sentence, and the method is not particularly limited here.
[0013]
The related word extraction unit 32 extracts a co-occurrence relationship between words, extracts words related to one word, and registers them in a co-occurrence frequency table as shown in FIG. 7 (step 104). The co-occurrence relationship here means words used together in one sentence. The co-occurrence frequency in the co-occurrence frequency table in FIG. 7 represents the number of times word 1 and word 2 are used together in one sentence. Regarding the extraction of co-occurrence relationships, not only words that appear in the same sentence but also general parsing methods can be used to determine the relationship between the subject, predicate, and dependency, but this method is particularly limited. do not do. Based on the extracted word and the co-occurrence relation, the strength of the relation between the words is obtained, a relation degree table as shown in FIG. 8 is created, and the result is registered in the related word dictionary 40 (step 105). The co-occurrence frequency may be the strength (relevance) between words, or the mutual information amount shown in FIG. 9 known as a method for obtaining the strength of the relationship between words may be the strength. In the present embodiment, the mutual information amount between words is assumed to be strength (relevance).
[0014]
FIG. 10 is a flowchart showing a process of creating the related word list 52 by the related word list creating unit 51 of FIG. The related word list creation unit 51 creates a related word list 52 of two keywords specified by the user. First, the first keyword input from the input device 60 of FIG. 2 is substituted into the variable A, the distance is substituted into the variable D, the relevance is substituted into the variable R, and the initial distance 0 is substituted into the variable D1 (step 201). Is used as an argument to call a related word list creation function (step 202) to create a related word list of the first keyword as shown in FIG.
[0015]
In FIG. 11, for convenience of explanation, the first keyword is a keyword A, and related words of the keyword A are related words indicated by symbols such as a related word A1 and a related word A2. The related word list in FIG. 11 is composed of a collection of data in units of lines. The list of each line is an arrangement of a head element and a list of related terms related to the head element. For example, [Keyword A, [Related Word A1, Related Word A2, Related Word A3, Related Word A4]] in the first line list in FIG. A2, related word A3, related word A4]. The data in the second and subsequent lines has the same expression format, and the related word list of the second keyword (keyword B) in FIG. 12 described later also has the same expression format. Further, in addition to calling the entire data of FIG. 11 and FIG. 12 as a “related word list”, for convenience of explanation, a list of related words related to the head element of each line of FIG. 11 and FIG. 12 is also called a “related word list”. And For example, in [Keyword A, [Related Word A1, Related Word A2, Related Word A3, Related Word A4]] which is the list on the first line in FIG. 11, a list in which related words related to Keyword A are arranged. [Related Word A1, Related Word A2, Related Word A3, Related Word A4] is also referred to as “related word list”.
[0016]
After the related words list of the first keyword is created in steps 201 and 202, the second keyword input from the input device 60 of FIG. An initial distance 0 is substituted into D1 for R (step 203), and a related word list creation function is called using these as arguments (step 204) to create a related word list of the second keyword as shown in FIG. . In FIG. 12, for convenience of explanation, the second keyword is a keyword B, and related words of the keyword B are related words B1 and related words B2, and related words are indicated by symbols.
[0017]
FIG. 13 is a flowchart showing the process of the related word list creation function called at step 202 and step 204 of FIG. In the related word list creation function, a related word list is created within a range of the specified relevance and distance. First, it is determined whether the distance D1 input as an argument is equal to or less than the specified distance D (step 301). If the distance D1 (number of related words) has already been exceeded, the process returns. If the distance D1 is within the specified distance D, it is checked whether the search for the related word of the keyword X input as an argument has been completed (step 302). In this search, the keyword X is searched from the relevance table of FIG. If there is a related word of the keyword X to be searched, the related word of X is acquired and substituted into X1 (step 303). Further, the degree of association between X and X1 is substituted into R1 (step 304). It is determined whether the relevance level R1 is equal to or less than the specified relevance level R (step 305). If the relevance level R1 is equal to or lower than the specified relevance level R, the process returns to step 302 without taking X1 as a related word. Process to get the next related word of. If R1 is a relevance level equal to or higher than the relevance level R in step 305, a process of adding X1 to the related word list of X is performed (step 306), and the process returns to step 302 to acquire the next related word of X I do. In this way, the related words of X are checked, and a related word list of X is created.
[0018]
When the search for the related word of X is completed, the same process is repeated for each element of the related word list of X. That is, for each element in the X related word list, a process for searching for and acquiring related words related to the element is performed. First, it is checked whether or not the search for the related word list of X is completed (step 307). If the search for the related word list of X has been completed, the process returns. If the search of the related word list of X is not completed, 1 is added to the distance D1 (step 308), and it is checked whether the search of each element of the related word list of X is completed (step 309). If the search is complete, return. If the search has not ended, an element (an element that has not been searched yet) is taken out from the related word list of X and assigned to Y (step 310). Then, the related word list creation function is recursively called with the element Y of the related word list of X, the specified distance D, the specified relevance level R, and the variable D1 as arguments (step 311). The above processing is performed to create a related word list of the first keyword shown in FIG. 11 and a related word list of the second keyword shown in FIG.
[0019]
FIG. 14 is a flowchart showing the process of the related word path creation unit 53 that creates the related word path list 54 based on the related word lists of FIGS. 11 and 12. First, the common related word list shown in FIG. 15 is created from the related word list of the first keyword (keyword A) shown in FIG. 11 and the related word list of the second keyword (keyword B) shown in FIG. 401). Next, a partial list from the element on the left side of the common related word list shown in FIG. 15 to the keyword A shown in FIG. 16 is created (step 402). Further, a partial list from the element on the right side of the common related word list shown in FIG. 15 to the keyword B shown in FIG. 17 is created (step 403). 16 and FIG. 17 are used, the related word reaching the keyword A shown in FIG. 18 by using the partial list from the element on the left side of the common related word list shown in FIG. 15 to the keyword A shown in FIG. A route list is created (step 404). Further, a related word path list reaching the keyword B shown in FIG. 19 is created using the partial list extending from the element on the right side of the common related word list shown in FIG. 15 to the keyword B shown in FIG. 17 (step 405). . Finally, the related word path list leading to keyword A and the related word path list leading to keyword B are combined to create a related word path list from keyword A to keyword B as shown in FIG. 20 (step 406). .
[0020]
Hereinafter, details of the processing of each step of FIG. 14 will be described in order.
[0021]
FIG. 21 is a flowchart showing the process of creating the common related word list in step 401 of FIG. First, it is checked whether the search of the related word list (FIG. 11) of the keyword A, which is the first keyword, is completed (step 501). If the search is completed, the process ends. If the search has not ended, it is checked whether the search for each element (keyword) in the related word list of keyword A has ended (step 502). If the search has ended, the process returns to step 501. . If the search has not ended, the next element is extracted from the related word list of keyword A and substituted for X (step 503), and the process proceeds to step 504.
[0022]
Note that step 501 checks whether the processing has been completed for all the row data when the processing is carried out using each row data of the related word list shown in FIG. 11 as a processing unit. That is, when the process is started from the first line data of the related word list in FIG. 11 and one line data is processed, the process proceeds to step 502 with the next line data as a processing target in step 501. If there is no more row data to be processed, the process ends from step 501. In step 502, it is checked whether the processing has been completed for all elements of the related word list in the row data to be processed. The elements extracted in step 503 are each element of the related word list in the row data to be processed.
[0023]
If the search for all the elements of the related word list in the current row data to be processed is not completed in step 502, the next element is extracted and substituted for X (step 503). It is checked whether the search of the list (FIG. 12) has been completed (step 504). If the search for the related word list of keyword B has been completed, the process returns to step 502. If the search for the related word list of keyword B has not been completed, it is checked whether the search for each element (keyword) in the related word of keyword B has been completed (step 505). If the search for each element (keyword) in the related word of keyword B has been completed, the process returns to step 504. If the search for each element (keyword) in the related word of keyword B has not been completed, each element (keyword) is extracted from the related word list of keyword B and substituted for Y (step 506). It is determined whether Y is the same (step 507). If X and Y are the same, a common related word list shown in FIG. 15 is created with the first element (keyword) of the matched Y related word list and the keyword X ([first element of the related word list of X, Y]). List format), the process returns to step 506. If X and Y are not the same, the process returns to step 505.
[0024]
Steps 504, 505, and 506 are the same processes as steps 501, 502, and 503, respectively. However, the related word list to be processed is the related word list of the keyword B in FIG. In addition, the variable to which the element is substituted in step 506 is Y.
[0025]
For example, the elements in the related word list of the keyword A in FIG. 11 and also included in the related word list of the keyword B in FIG. 12 are the related word A3, the related word A11, and the related word A12. Those head elements (the head element in the related word list of FIG. 12) are related words B3 for related word A3, related words B2 and related words B11 for related word A11, and related words B1 for related word A12. Since it is the word B11, the list is as shown in FIG. The element on the left side of each row data in the common related word list in FIG. 15 is the right side of each row data in the related word list in FIG. 12 among the elements in the related word list on the right side of each row data in the related word list in FIG. Has the same elements as in the related word list. Further, the element on the right side of each row data of the common related word list in FIG. 15 is the head element corresponding to the element on the left side (head element in the related word list in FIG. 12).
[0026]
FIG. 22 is a flowchart showing a process of creating a partial list reaching keyword A in step 402 in FIG. First, it is checked whether the search for the common related word list in FIG. 15 is completed (step 601). If the search is completed, the process is terminated. That is, when the process is started from the first row data of the common related word list in FIG. 15 and one row data is processed, the next row data is processed in step 601 and the process proceeds to step 602. If there is no more row data to be processed, the process ends from step 601. If the search is not completed in step 601, the element on the left side of the row data to be processed in the common related word list in FIG. 15 is extracted and substituted for X (step 602). It is checked whether or not (step 603). If X is the same as keyword A, the process returns to step 601 to perform processing for extracting the next row data of the common related word list in FIG. 15 as a processing target.
[0027]
If X is not the same as keyword A in step 603, the top element of the related word list including X in the related word list of keyword A in FIG. 11 is substituted for Y (step 604). That is, the line data including X in the related word list on the right side in each line data in FIG. 11 is found, and the head element is substituted for Y. Next, it is checked whether or not the partial lists of Y and X have already been created (step 605). If they have been created, the process returns to step 601. If it has not been created, a partial list of Y and X is created (step 606). This partial list is in the format [X, Y]. Next, it is checked whether Y is a keyword A, that is, a terminal keyword (step 607). If Y is the same as keyword A, the process returns to step 601. If Y is not the same as keyword A, Y is substituted for X (step 608), and the process returns to step 604 to reach the keyword A shown in FIG. Repeat the process of creating a partial list.
[0028]
For example, when searching the related word list of the keyword A in FIG. 11 using the related word A11 which is the element on the left side as a key in the second row list in the common related word list in FIG. , [Related word A11, related word A12, related word A13]] and [related word A2, [related word A21, related word A22, related word A23, related word A11]], and their head elements are related. Since the word A1 and the related word A2, [related word A11, related word A1] and [related word A11, related word A2] are created as a partial list leading to the keyword A.
[0029]
FIG. 23 is a flowchart showing a process of creating a partial list reaching keyword B in step 403 of FIG. First, it is checked whether the search for the common related word list in FIG. 15 is completed (step 701). If the search is completed, the process is terminated. That is, when the process is started from the first row data of the common related word list in FIG. 15 and one row data is processed, the next row data is processed in step 701 and the process proceeds to step 702. When there is no more row data to be processed, the process ends from step 701. If the search is not completed in step 701, the element on the right side of the row data to be processed in the common related word list in FIG. 15 is extracted and substituted for X (step 702). It is checked whether or not (step 703). If X is the same as keyword B, the process returns to step 701, and the next row data of the common related word list in FIG.
[0030]
If X is not the same as keyword B in step 703, the top element of the related word list including X in the related word list of keyword B in FIG. 12 is substituted for Y (step 704). That is, row data including X in the related word list on the right side in each row data in FIG. 12 is found, and the head element is substituted for Y. Next, it is checked whether or not the partial lists of Y and X have already been created (step 705). If they have been created, the process returns to step 701. If it has not been created, a partial list of Y and X is created (step 706). This partial list is in the format [X, Y]. Next, it is checked whether Y is a keyword B, that is, a terminal keyword (step 707). If Y is the same as keyword B, the process returns to step 701. If Y is not the same as keyword B, Y is substituted for X (step 708), and the process returns to step 704 to reach keyword B shown in FIG. Repeat the process of creating a partial list.
[0031]
For example, when searching the related word list of the keyword B in FIG. 12 using the related word B11 which is the element on the right side in the list in the third row in the common related word list in FIG. 15, [Related Word B1 , [Related Word B11, Related Word A12, Related Word B12, Related Word B13]] and [Related Word B2, [Related Word B21, Related Word B11, Related Word A11]], and their head elements are related. Since it is the word B1 and the related word B2, [Related Word B11, Related Word B1] and [Related Word B11, Related Word B2] are created as a partial list leading to the keyword B.
[0032]
FIG. 24 is a flowchart showing a process of creating a related word path list that reaches keyword A in step 404 of FIG. First, it is checked whether or not the search for the common related word list in FIG. 15 has been completed (step 801). If the search has been completed, the process is terminated. That is, if the process is started from the first row data of the common related word list in FIG. 15 and one row data is processed, the next row data is processed in step 801 and the process proceeds to step 802. When there is no more row data to be processed, the process ends from step 801. If the search is not completed in step 801, the element on the left side of the row data to be processed in the common related word list in FIG. 15 is extracted and substituted for X (step 802), and a related word path list for X is created. (Step 803). The first related word path list is a list [X] having only X as an element. Next, X is substituted into X1 (step 804), and it is checked whether the search for the partial list reaching the keyword A in FIG. 16 is completed (step 805). In this search, a list in which the element on the left side matches X1 is searched from the partial list reaching the keyword A in FIG.
[0033]
When the search for the partial list leading to the keyword A in FIG. 16 has been completed, the process returns to step 801 to perform processing for retrieving the next row data of the common related word list in FIG. 15 as a processing target. If the search is not completed, the right element Y whose left element matches X1 in the partial list leading to keyword A in FIG. 16 is obtained (step 806), and Y is added to the related word path list of X. Add (step 807). This is a process of inserting Y as the head element of the related word path list of X. Next, it is checked whether Y is the keyword A, that is, the end keyword (step 808). If it is the same as the keyword A, that is, if it is the end keyword, the process returns to step 801, and if Y is not the same as the keyword A, , Y is substituted into X1 (step 809), and the process returns to step 806 to repeat the process of creating the related word path list reaching the keyword A shown in FIG.
[0034]
For example, when searching the partial list reaching the keyword A in FIG. 16 using the related word A12 as the left element in the list in the fourth row in the common related word list in FIG. 15, the left element is The list that matches the related word A12 includes [related word A12, related word A1] and [related word A12, related word A11], and when looking at the former [related word A12, related word A1], Since the element on the right side of the word A12, the related word A1] is the related word A1, first, the related word A1 is added to the related word path list [related word A12] of the related word A12 to create [related word A1, related word A12]. To do. Further, since the related word A1 is not the keyword A, that is, the terminal keyword, further searching the partial list leading to the keyword A finds a list [related word A1, keyword A] whose left side element matches the related word A1, The element keyword A on the right side is added to the related word list of the related word A12 to create a list [keyword A, related word A1, related word A11].
[0035]
FIG. 25 is a flowchart showing a process of creating a related word path list reaching keyword B in step 405 of FIG. First, it is checked whether the search for the common related word list in FIG. 15 is completed (step 901). If the search is completed, the process is terminated. That is, if the process is started from the first row data of the common related word list in FIG. 15 and one row data is processed, the process proceeds to step 902 with the next row data as a processing target in step 901. If there is no more row data to be processed, the process ends from step 901. If the search is not completed in step 901, the element on the right side of the row data to be processed in the common related word list in FIG. 15 is extracted and substituted for X (step 902), and a related word path list for X is created. (Step 903). The first related word path list is a list [X] having only X as an element. Next, X is substituted into X1 (step 904), and it is checked whether the search for the partial list reaching the keyword B in FIG. 17 is completed (step 905). In this search, a list in which the element on the left side matches X1 is searched from the partial list that reaches keyword B in FIG.
[0036]
If the search for the partial list reaching keyword B in FIG. 17 has been completed, the process returns to step 901 to perform processing for extracting the next row data of the common related word list in FIG. 15 as a processing target. If the search has not ended, the right element Y whose left element matches X1 in the partial list leading to the keyword B in FIG. 17 is obtained (step 906), and Y is added to the related word path list of X. It adds (step 907). This is a process of inserting Y as the last element of the related word path list of X. Next, it is checked whether or not Y is a keyword B, that is, a terminal keyword (step 908). If it is the same as keyword B, that is, if it is a terminal keyword, the process returns to step 901, and if Y is not the same as keyword B , Y is substituted for X1 (step 909), the process returns to step 906, and the process of creating the related word path list reaching the keyword B shown in FIG. 19 is repeated.
[0037]
For example, when a partial list reaching the keyword B in FIG. 17 is searched using the related word B11 as the right element in the list in the third row in the common related word list in FIG. 15, the left element is The list that matches with the related word B11 includes [related word B11, related word B1] and [related word B11, related word B2], and when looking at the former [related word B11, related word B1], Since the element on the right side of the word B11, the related word B1] is the related word B1, first, the related word B1 is added to the related word path list [related word B11] of the related word B11 to create [related word B11, related word B1]. To do. Further, since the related word B1 is not the keyword B, that is, the terminal keyword, when a partial list reaching the keyword B is further searched, a list of [related word B1, keyword B] whose left side element matches the related word B1 is found. The element keyword B on the right side is added to the related word list of the related word B11 to create a list [related word B11, related word B1, keyword B].
[0038]
FIG. 26 is a flowchart showing the process of creating a related word path list from keyword A to keyword B in step 406 of FIG. First, it is checked whether or not the search for the common related word list in FIG. 15 has been completed (step 1001). If the search has been completed, the process is terminated. That is, if processing is started from the first row data of the common related word list in FIG. 15 and processing is performed for one row data, the processing proceeds to step 1002 with the next row data as a processing target in step 1001. If there is no more row data to be processed, the process ends from step 1001. If the search is not completed in step 1001, the element on the left side of the row data to be processed in the common related word list in FIG. 15 is extracted and substituted for X (step 1002), and the element on the right side is extracted. Substitute for Y (step 1003). Next, it is checked whether the search for the related word path list leading to the keyword A in FIG. 18 is completed (step 1004). In this search, a list in which the rightmost element coincides with X is searched from the related word path list reaching the keyword A in FIG.
[0039]
When the search for the related word path list reaching the keyword A in FIG. 18 has been completed, the process returns to step 1001 to perform processing for extracting the next row data of the common related word list in FIG. 15 as a processing target. If the search is not completed, a list in which the rightmost element matches X in the related word path list leading to keyword A in FIG. 18 is acquired and substituted into L1 (step 1005). Subsequently, it is checked whether or not the search for the related word path list leading to the keyword B in FIG. 19 is completed (step 1006). In this search, a list in which the leftmost element matches Y is searched from the related word path list reaching the keyword B in FIG.
[0040]
If the search for the related word path list reaching the keyword B in FIG. 19 has been completed, the process returns to step 1004. If the search has not been completed, the related word path list in the related word path list reaching the keyword B in FIG. A list in which the leftmost element matches Y is acquired and assigned to L2 (step 1007). Then, the list of L1 and the list of L2 are combined to form a related word path list from the keyword A to the keyword B (step 1008). After combining the lists, the process returns to step 1006 to repeat the process of combining with other lists.
[0041]
For example, looking at the list [related word A3, related word B3] on the first line in the common related word list in FIG. 15, the element on the left side is related word A3, which leads to keyword A in FIG. In the related word list, a list whose right side element matches the related word A3 is [keyword A, related word A3]. Further, the element on the right side of the list [Related Word A3, Related Word B3] on the first row is the related word B3, and the left element in the related word list leading to the keyword B in FIG. 19 matches the related word B3. The list to be performed is [Related Word B3, Keyword B]. Therefore, these two lists are combined to create a list [keyword A, related word A3, related word B3, keyword B]. The above processing is performed to create a related word path list from the first keyword (keyword A) to the second keyword (keyword B) as shown in FIG. In FIG. 20, each list of lines represents a route from the keyword A to the keyword B.
[0042]
FIG. 27 is a flowchart showing a process of creating display data by the related word path display unit 55 of FIG. The related word path display unit 55 creates display data based on the related word path list 54 (FIG. 20). First, the average value of the relevance of each route is calculated (step 1101), and the display order is determined (step 1102). The average value of the degree of association of each route is the average value of the degree of association between adjacent elements in each route list. The display order is the order specified by the user. In the shortest path order, the list is displayed in order from the list with the smallest number of elements in the related word path list, and in the longest path order, the list is displayed in order from the list with the largest number of elements in the related word path list. If the average value of the related strengths is in descending order, they are displayed in descending order of the average value calculated in step 1101. Further, the display order is determined by a combination thereof. When the route to be displayed is determined, display data is created (step 1103). Optionally, it can be specified to change the display color of routes below a predetermined relevance. In addition, it is possible to specify that the display color of a keyword whose appearance frequency is equal to or less than a predetermined value is changed. For example, when the display color of a route having a relevance level of 3.0 or less is designated as red, the route between the corresponding keywords is designated as red. In addition, when the display color of related words having an appearance frequency of 50 or less is designated as blue, the background of the corresponding keyword is designated as blue.
[0043]
FIG. 28 shows an example of display data creation in the HTML (HyperText Markup Language) format. FIG. 28 shows an example in which the route between “character” and “difficult to see” is designated in red and the background of “difficult to see” is designated in blue. As described above, display data is created according to the designated option, and the search result is displayed on the input / output screen 80 of the display device 70 of FIG.
[0044]
In addition, although the example which changes a display color was demonstrated here, the display mode to change is not restricted to a display color. For example, you may make it change the thickness of a line, an emphasis display, the presence or absence of a blink, etc.
[0045]
FIG. 29 shows a display example of search results. In this example, “personal computer” and “elderly” are input as keywords and searched. From the example of FIG. 29, there are many keywords “Internet”, “Mail”, “Motivation to learn” between “PC” and “Elderly”, “Difficult”-“Motivation to learn” and “Text”-“Difficult to see” ”Is highlighted in red according to the user ’s designation, and the keyword“ difficult to see ”is highlighted in blue. And it is better to enlarge the display characters on the display. Thus, by displaying related words between keywords, it is possible to obtain information that could not be obtained by a conventional network-type display method.
[0046]
The present invention is not limited to the embodiment described with reference to FIGS. 1 to 29, and various modifications can be made without departing from the scope of the invention. For example, in the above-described embodiment, an example of displaying in the HTML format as the display method has been described. However, it is also possible to create and display a graphic image.
[0047]
According to the above embodiment, since a plurality of related word paths from the first keyword to the second keyword are displayed, it is possible to grasp what word the two words are connected to. By displaying the path of such a related word, it is possible to present unknown information that the user has not noticed until now. In addition, since it is possible to change the display method with the shortest path order, the longest path order, the ascending order of the average value of the related strengths, and the descending order, it is possible to see the connection between words from different viewpoints. Discover new knowledge not only by judging trends in various keywords such as daily reports, store manager's diaries, and general newspaper data, but also by observing the connections. It becomes possible to do. For example, it is possible to search for potential needs even when sufficient questionnaires are not available, such as what functions are necessary for product development such as a PC for elderly people and a mobile phone for elderly people.
[0048]
【The invention's effect】
As described above, according to the present invention, by displaying a plurality of related word paths from the first keyword to the second keyword in advance, it is possible to determine which word the two keywords are connected to. By presenting information and displaying the path of such related terms, it is possible to present unknown information that the user has not noticed before.
[Brief description of the drawings]
FIG. 1 is a diagram showing association relationships between words in a network format.
FIG. 2 is a diagram showing a configuration of a text mining device according to an embodiment of the present invention.
FIG. 3 is a flowchart showing a processing process until a related word dictionary is created from a document database.
FIG. 4 is a diagram showing an example of document data read from a document database.
FIG. 5 is a diagram illustrating an example of a word that is analyzed and cut out by a word extraction unit;
FIG. 6 is a diagram showing an example of a word table created by a word extraction unit.
FIG. 7 is a diagram showing an example of a co-occurrence frequency table created by a related word extraction unit.
FIG. 8 is a diagram showing an example of a relevance level table created by a related word extraction unit.
FIG. 9 is a diagram showing a calculation formula for mutual information used in the relevance degree table creation process;
FIG. 10 is a flowchart showing a process of creating a related word list by a related word list creating unit.
FIG. 11 is a diagram illustrating an example of a related word list of a first keyword.
FIG. 12 is a diagram illustrating an example of a related word list of a second keyword.
FIG. 13 is a flowchart showing a process of a related word list creation function.
FIG. 14 is a flowchart showing a process of creating a related word path list by a related word path creating unit.
FIG. 15 is a diagram showing an example of a common related word list of the related word list of the first keyword and the related word list of the second keyword.
FIG. 16 is a diagram showing an example of a partial list that reaches the first keyword (keyword A);
FIG. 17 is a diagram showing an example of a partial list that reaches the second keyword (keyword B);
FIG. 18 is a diagram showing an example of a related word path list that reaches the first keyword (keyword A);
FIG. 19 is a diagram showing an example of a related word path list that reaches the second keyword (keyword B);
FIG. 20 is a diagram showing an example of a related word path list from a first keyword (keyword A) to a second keyword (keyword B);
FIG. 21 is a flowchart showing a process of creating a common related word list.
FIG. 22 is a flowchart showing a process of creating a partial list up to the first keyword (keyword A).
FIG. 23 is a flowchart showing a process of creating a partial list up to the second keyword (keyword B).
FIG. 24 is a flowchart showing a process of creating a related word path list up to the first keyword (keyword A).
FIG. 25 is a flowchart showing a process of creating a related word path list up to the second keyword (keyword B).
FIG. 26 is a flowchart showing a process of creating a related word path list from the first keyword (keyword A) to the second keyword (keyword B).
FIG. 27 is a flowchart showing a process of creating display data in a related word path display unit.
FIG. 28 is a diagram showing an example of creating display data in HTML format.
FIG. 29 is a diagram showing a search result of a related word list from the first keyword to the second keyword.
[Explanation of symbols]
10 …… Processing device
20 …… Document database
30 …… Related word dictionary creation part
31 …… Word extractor
32 …… Related word extractor
40 …… Related word dictionary
50 …… Related word path search unit
51 …… Related word list creation part
52 …… Related word list
53 …… Related word path creation part
54 …… Related word path list
55 …… Related word path display
60 …… Input device
70 …… Display device
80 …… Input / output device

Claims (6)

A text mining program for analyzing text data, understanding features and trends, and discovering unknown information.
On the computer,
Searching for a path of related terms connecting two specified keywords;
A text mining program for executing a step of displaying the searched route in the shortest route order or the longest route order according to a user's specification . A text mining program for analyzing text data, understanding features and trends, and discovering unknown information.
On the computer,
Searching for a path of related terms connecting two specified keywords;
A text mining program for executing a step of displaying the searched route in ascending or descending order of the average value of the strength of association between the keywords as the elements of the route according to the user's specification . In a text mining device constructed using a computer, it is a text mining method for analyzing text data, grasping features and trends, and discovering unknown information,
A route search means provided in the computer for searching for a route of related words connecting the two specified keywords;
Text mining method which the display unit included in the computer, in response to user's designation, the shortest path order or longest path order, and executes the step of displaying the searched route. In a text mining device constructed using a computer, it is a text mining method for analyzing text data, grasping features and trends, and discovering unknown information,
A route search means provided in the computer for searching for a route of related words connecting the two specified keywords;
The display means provided in the computer executes a step of displaying the searched route in ascending or descending order of the average value of the strength of association between the keywords that are the elements of the route according to the user's designation. A featured text mining method. A text mining device for analyzing text data, understanding features and trends, and discovering unknown information,
Means for searching a path of related terms connecting two specified keywords;
A text mining device comprising: means for displaying searched routes in the shortest route order or in the longest route order according to a user's specification . A text mining device for analyzing text data, understanding features and trends, and discovering unknown information,
Means for searching a path of related terms connecting two specified keywords;
A text mining device comprising: means for displaying the searched routes in ascending or descending order of the average value of the strength of association between the keywords that are elements of the route according to the user's specification .

Images