JP5450699B2 - Document analysis apparatus and document analysis program - Google Patents

Description

  Embodiments described herein relate generally to a document analysis apparatus and a document analysis program.

  With the advancement of information systems in recent years, it has become possible to record and accumulate a large amount of documents such as patent documents, newspaper articles, web pages, and books. Here, there is a need for a technique for classifying and analyzing these accumulated document groups and effectively utilizing them for daily work. For example, a huge amount of past newspaper articles are classified into categories and arranged so that many people can use them easily, or patents of other companies related to the technology currently being researched and developed. Analyzing trends and discovering new R & D fields.

  As described above, in a category in which a large number of documents are classified, there is a need to know the characteristics of the category in order to grasp the document contents in the category.

  As a technique for examining the characteristics of a category, for example, there is a method of statistically extracting feature words representing differences between document sets.

  However, this method does not extract in consideration of the ranking given to the category. For example, in the defect information of automobiles, there is a document set classified into categories ranked as “extremely dangerous”, “slightly dangerous”, “dangerous”, and “safety” in the order of the risk of failure. At this time, the category with the first risk level is “extremely dangerous”, the category with the second risk level is “slightly dangerous”, the category with the third risk level is “danger”, and the category with the fourth risk level is Is "safe".

  At this time, if you want to know the factors that classified the document data into the category of “danger”, that is, the factors that gave the document data a ranking of “danger”, the statistical method as described above is not necessarily ranked. There is a problem that it is not always possible to extract words that cause the problem.

  For example, in the above-described vehicle defect information, the factors classified into the “very dangerous” category in the document set classified into the ranking categories “very dangerous”, “dangerous”, and “safety” according to the risk level of the failure are as follows. , "Fuel", "Leakage", "Tire", "Dropout", if you extract the feature word of the category by the statistical method as above, like "pump", "pipe", Words that appear frequently in a category are extracted, and words that are not “very dangerous” may be extracted.

JP 2007-265068 A JP 2007-241905 A

  The problem to be solved by the present invention is to provide a document analysis apparatus and a program for extracting words that cause an increase or decrease in category ranking.

The document analysis apparatus according to the embodiment includes a document storage unit that stores document data, a plurality of categories with rankings that classify document data, and a classification result storage unit that stores a hierarchical structure of categories, and one of the plurality of categories. In one category, from the word group included in the document data classified into the category, the appearance frequency of the word in the category is higher than the appearance frequency of the word in other categories in the same hierarchy as the category, and A factor word extraction unit that extracts words whose frequency of appearance in other categories decreases as the rank becomes farther from the category.

FIG. 2 is a block diagram showing a hardware configuration of the document analysis apparatus according to the embodiment. FIG. 2 is a block diagram showing a functional configuration of the document analysis apparatus according to the embodiment. 4 is an exemplary block diagram illustrating an example of document data stored in a document storage unit of the document analysis apparatus according to the embodiment. FIG. The block diagram which shows an example of the category memorize | stored in the classification result memory | storage part of the document analyzer which concerns on embodiment. 6 is a flowchart illustrating an example of an operation of the document analysis apparatus according to the embodiment. FIG. 10 is a diagram showing a display example of classification results of the document analysis apparatus according to the embodiment. The figure which shows the example of a display when the set of the category with a rank of the document analyzer which concerns on embodiment is input. 6 is a flowchart illustrating an example of processing of a factor word extraction unit of the document analysis apparatus according to the embodiment. The conceptual diagram of the factor word extraction process by the factor word extraction part of the document analyzer which concerns on embodiment. The conceptual diagram of the factor word extraction process of the factor word extraction part of the document analyzer which concerns on embodiment. The figure which shows the example of an extraction result of the factor word extraction part of the document analyzer which concerns on embodiment. 6 is a flowchart illustrating an example of processing of a complex factor word extraction unit of the document analysis apparatus according to the embodiment. 6 is a flowchart illustrating an example of processing of a complex factor word extraction unit of the document analysis apparatus according to the embodiment. The figure which shows the example of an extraction result of the compound factor word extraction part of the document analyzer which concerns on embodiment. 5 is a flowchart illustrating an example of processing of a factor category generation unit of the document analysis apparatus according to the embodiment. The figure which shows an example of the process result of the factor category production | generation part of the document analyzer which concerns on embodiment. 6 is a flowchart showing an example of processing of a biaxial map visualization unit of the document analysis apparatus according to the embodiment. The figure which shows the example of a display of the result of the process of the biaxial map visualization part of the document analyzer which concerns on embodiment.

(First embodiment)
The document analysis apparatus according to the embodiment will be described below with reference to FIGS. The following apparatus can be implemented with either a hardware configuration or a combined configuration of hardware resources and software. As the software of the combined configuration, a program that is installed in advance on a computer of a corresponding device from a network or a storage medium and that realizes the function of the corresponding device is used.

  FIG. 1 is a schematic diagram illustrating a configuration of a document analysis apparatus 10 according to the present embodiment. The computer 10 includes a main storage device 11, a keyboard 12, a mouse 13, a central processing unit 14, and a display device 15. The document analysis apparatus 10 is realized by, for example, a computer to which the document analysis program according to this embodiment is applied.

  The storage device 11 is a storage device that can be read / written from the central processing unit 14, and is, for example, a RAM. The storage device 11 stores the document analysis program of this embodiment in advance.

  The keyboard 12 and the mouse 13 are normal input devices. For example, various information including data or commands is input to the central processing unit 15 by an operation of the operator.

  The central processing unit 14 is a normal CPU, and controls the execution of each program based on, for example, a function for executing a document analysis program stored in the storage device 11 and information input from the keyboard 12 or the mouse 13. A function and a function of outputting an execution result or the like to the display device 15;

  The display device 15 is a normal display device. For example, the display device 15 displays and visualizes each architecture model and feature model being edited, and has a function of displaying information output from the central processing unit 14.

  With reference to FIG. 2, the functional configuration of the document analysis apparatus 10 configured as described above will be described.

  The document analysis apparatus 10 includes a document storage unit 1, a classification result storage unit 2, a user interface unit 3, and a factor generation unit 4.

  The document storage unit 1 stores data of a plurality of documents (hereinafter referred to as document data) to be analyzed by the document analysis apparatus 10. In the present embodiment, the document data stored in the document storage unit 1 is document data related to vehicle malfunction information, and is classified by a plurality of categories configured in a hierarchical structure.

  Here, an example of document data stored in the document storage unit 1 will be described with reference to FIG. The document data 20 stored in the document storage unit 1 is document data for reporting vehicle malfunction information, and includes a document number 21, a title 22, a body 23, a malfunction risk 24, a development company 25, and a purchase date 26. As items, the contents of document data corresponding to each item are stored.

  The document number 21 indicates an ID unique to this document data. The title 22 is the title of the document data 20. The body 23 is the body of the document data 20. Defect risk 24 indicates the risk of defect information reported by the document data. The development company 25 is a development company of a car in which a failure has occurred, and purchase date 26 is a date on which a car in which a failure has occurred is purchased.

  Note that the item names of the document data 20 need not be all of the above item names, and other item names may be added. Although one document data is shown here as an example, the document storage unit 1 may store a plurality of document data.

  The classification result storage unit 2 stores a category in the document data stored in the document storage unit 1 and a hierarchical structure of the category. In this embodiment, it is assumed that the categories stored in the classification result storage unit 2 are created in advance and the document data is classified. The category may be created by clustering document data.

  With reference to FIG. 4, the category memorize | stored in the classification result memory | storage part 2 and the hierarchical structure of this category are demonstrated. The categories (a) to (f) shown in FIG. 4 include a category number 301, a parent category 302, a category name 303, and a document 304 as items.

  The category number 301 stores unique data for the document analysis apparatus 10 to specify a category.

  The parent category 302 stores data indicating the parent category of this category. That is, the parent category 302 is the category number of the parent category of the category. For example, the parent category of category (a) is “(none)” because category (a) is located at the top (root) of the hierarchical structure in categories (a) to (f).

  The parent category 302 of the category (b) (category number “c02”) is the category (category (a)) of the category number “c01”. That is, it can be said that one of the child categories of category (a) is category (b). Similarly, the parent category 302 is set for the categories (c) to (f).

  Hereinafter, in the present embodiment, a category located at a direct parent of a category is called a parent category, and a category located at a direct child is called a child category. A category that is located in the direct or indirect parent of a certain category is generally called an upper category, and a category that is located in a direct or indirect child (descendant) of a certain category is generally called a lower category.

  The category name 303 stores a name for each category to indicate its contents to the user.

  The document 304 stores the document number in the document data included in the category. When a plurality of documents belong to a category, a plurality of document number data are listed.

  Note that, as shown in category (c), a condition 305 to be satisfied by a document classified into the category may be set, and only document data satisfying the condition stored in the condition 305 may be listed in the document 304. In the categories (a), (b), and (d), the document data stored in the document 304 is “none”, that is, the documents that belong to each category are “(none)”. . This means that there is no document that is directly classified into the category, and there may be a document that is indirectly classified through the lower category. For example, a document that is indirectly classified in category (a) is a union of documents that are classified in all lower categories of category (a).

  The user interface unit 3 includes a category display operation unit 31 and an axis map visualization unit 32, and receives user input information and instruction information.

  The category display operation unit 31 displays a screen showing the categories stored in the classification result storage unit 2 and its hierarchical structure to the user (hereinafter referred to as a category display screen). The category display operation unit 31 also has a function as an input unit that receives a user operation on the displayed category display screen. The category display screen will be described later.

  The biaxial map visualization unit 32 uses, as the first classification axis, all categories with ranks that are children categories of the category input by the user using the category display operation unit 31, and a factor word extraction unit 41 and a compound factor word described later. Based on the extraction result of the extraction unit 42, each category classified by the factor category generation unit 43 is used as a second classification axis, and the total number of documents crossed by each category is displayed on the two-axis map. .

  Next, the factor generation unit 4 will be described. The factor generation unit 4 includes a factor word extraction unit 41, a complex factor word extraction unit 42, and a factor category generation unit 43.

  When the user inputs a category through the category display operation unit 31, the factor word extraction unit 41 is a group of words that cause the category rank to rise or fall in the ranked category that is a child category belonging to the input category (first group). 1 word group) is extracted.

  When the words included in the first word group are combined in the first word group extracted by the factor word extraction unit 41, the compound factor word extraction unit 42 causes the category rank to rise or fall. Extract word combinations.

  Furthermore, the compound factor word extraction unit 42 similarly applies a category group (second word group) not extracted by the factor word extraction unit 41 when combining words included in the second word group. Extract word combinations that cause the ranking to rise or fall.

  The factor category generation unit 43 creates a new category based on the words and word combinations extracted by the factor word extraction unit 41 and the compound factor word extraction unit 42 and stores them in the classification result storage unit 2. In the new category, document data including the word and a combination of words is classified.

  The document storage unit 1 and the classification result storage unit 2 are realized using a file system, a database, or the like, which is a storage unit of a computer. The user interface unit 3 is realized by a keyboard 12, a mouse 13, and a display 15. The factor generation unit 430 is realized by the central processing unit 14 executing the program stored in the storage device 11.

  The following is document data relating to vehicle malfunction information stored in the document storage unit 1 shown in FIG. 3, and is ranked in the order of “very dangerous”, “somewhat dangerous”, “dangerous”, and “safety” in order of the risk of malfunction. When a plurality of document data is stored for each category, and the user wants to know the factor that the document data is classified into the category with the ranking of “danger”, that is, the factor of risk assignment, “danger” FIG. 5 to FIG. 18 show an example of the operation of the document analysis apparatus 10 that extracts a word indicating a factor classified into the category (hereinafter referred to as a factor word) from words included in the document data belonging to the ranking category. I will explain.

  FIG. 5 is a flowchart illustrating an example of the operation of the document analysis apparatus 10.

  First, a category display screen is displayed on the user interface unit 3 (step S401).

  FIG. 6 shows an example of the category display screen displayed in step S401. The category display screen 50 shown in FIG. 6 displays a hierarchical structure of categories in the left part 501 of the screen. When one of the categories is selected, a title list of documents belonging to the selected category is displayed in the upper right part 502 of the screen. Is done. Further, when one of the titles displayed in a list is selected, the text of sentence 2 data is displayed in the lower right 503 of the screen. In FIG. 6, the “extremely dangerous” category displayed in dots is selected as the category in which the user wants to check the document data, and the title list of the document data belonging to the “extremely dangerous” category is displayed in the upper right portion 502 of the screen. . Also, among the document data belonging to the “extremely dangerous” category displayed in the upper right portion 502 of the screen, “fuel leak” displayed as a dot is selected, and the text of the document data of “fuel leak” is displayed at the lower right 503 of the screen. Is displayed.

  Subsequently, among the parent categories displayed on the left part 501 of the category display screen 50 via the category display operation unit 31, the user analyzes factors for ranking the ranked categories belonging to the parent category (factor The target parent category from which words are extracted is input (step S402).

  FIG. 7 shows an example of the category display screen 50 when the parent category input in step S402 is input. As illustrated in FIG. 7, in step S <b> 402, the user uses the user interface unit 3 to select the “by defect risk level” category 601 as a parent category from which factor words are extracted. After that, when a message screen 602 with a message “Do you want to extract factors of each category immediately under the selected category?” Is displayed, the user presses a “YES” button 603 to cause factor word extraction processing described later. Is executed. If the “NO” button 604 is pressed, the process ends (not shown).

  When the parent category is input, the factor generation unit 4 performs factor generation processing (steps S403 and S404). The factor generation process includes a factor word extraction process that extracts a single word that causes a rise or fall in each ranking category directly under the parent category to be analyzed, and a rise or fall in rank in each ranking category. A compound factor word extraction process for extracting a factor word combination is performed.

  With reference to FIGS. 8 and 9, the factor word extraction process performed by the factor word extraction unit 41 in step S403 will be described. Here, a case will be described in which factor word extraction processing is performed for the “by defect risk level” category of FIG.

  First, the factor word extraction unit 41 initializes list1, which is a list of extraction results in the factor word extraction process (step S1101). For each category Ci with ranking (step S1102), it repeats for each word t in all documents of category Ci (step S1103), and determines whether the appearance frequency of t in Ci is statistically significant. (Step S1104).

  The determination of whether or not it is statistically significant is, for example, whether or not it is significant by a chi-square test. According to the chi-square test in this case, the chi-square statistic (χ) has a chi-square distribution (3.84) with a significance level of 5% and a chi-square distribution (3.84) with a significance level of 2 degrees ( If it is smaller than 6.63), it becomes statistically significant. On the other hand, when the chi-square statistic (χ) is larger than the chi-square distribution (3.84) with a significance level of 5% and the chi-square distribution (6.63) with a significance level of 1% and a significance level of 1%. Is not statistically significant. Note that the χ square statistic (χ) is calculated by the equation (1) shown in FIG.

  In Expression (1), x11 is the number of documents in which the word t in the category Ci appears. a1 is the number of documents in which the word t appears in all documents. b1 is the number of documents of category Ci. n is the number of all documents. X12 is a1-x11, x21 is b1-x11, and x22 is n-a1-x21.

  If it is not determined to be significant by this χ square test (No in step S1104), the process is repeated for other words. If it is determined to be significant (step S1104 is Yes), score (Ci, t) which is the score of the word t is obtained, and t and its score are substituted into list1 (step S1105). The score (Ci, t) will be described later. Note that score (Ci, t)) is calculated by equation (2) in FIG. However, the ranking category is (C1, C2,... Ci,..., Cm).

  In Equation (2), df (Ci, t) is the number of documents in which the word t in the category Ci appears, and D is the total document set. flag (i, j) is a function (variable) that takes −1 when i <j and +1 when i> j.

  Here, the concept of equation (2) shown in FIG. 9 will be described. In each category with ranking, the word that becomes a factor has a high appearance frequency of the word in the category, and the appearance frequency decreases as the rank rises or falls (decreases the rank) from the category. Based on such an idea, the above equation (2) is defined.

  First, the appearance frequency of the word t in the category Ci is obtained by df (Ci, t) / df (D, t) in the expression (2). Then, the degree of decrease in the appearance frequency of the adjacent categories is calculated by df (Cj + 1, t) −df (Cj, t), and the degree of decrease increases the weight in the category of the rank near the factor category. Log (m− | j + 1−i |) is applied. flag (i, j) is +1 for adding a score when the appearance frequency is rising for the rank before the factor category, and for the rank after the category, Use -1 to add a score when is moving down.

  FIG. 10 shows an example of words included in all document data belonging to the parent category “according to the risk level of defects” and the appearance frequency of each word in the child category of the category input in step S402. The items in the vertical direction of the table shown in FIG. 10 are words included in the document data, and the items in the horizontal direction are child categories of the category input in step S402.

  As shown in FIG. 10, “seat belt” has an appearance frequency of 39 in the “slightly dangerous” category, and is more frequent than other ranking categories. Furthermore, in the “slightly dangerous” category and the “extremely dangerous” category, the frequency reduction degree is reduced from 39 to 14, and the reduction degree from the “slightly dangerous” category to the “dangerous” category is also 39 to 12, “ The reduction from the “dangerous” category to the “extremely dangerous” category is also a result of 12 to 5, and the word “seat belt” has a high frequency of appearance in the “slightly dangerous” category and ranks from the “slightly dangerous” category. It is a word whose frequency of occurrence decreases with increasing distance.

  However, the word “fuel” has the highest frequency of occurrence in the “slightly dangerous” category compared to the frequency of appearance in the other ranked categories, but the degree of decrease from the “slightly dangerous” category to the “very dangerous” category. 32 to 14, the decrease from the “slightly dangerous” category to the “dangerous” category is 32 to 26, and the “dangerous” category to the “extremely dangerous” category increases from 26 to 28. That is, the word “fuel” is less reduced than the word “seat belt”, so the score of the “slightly dangerous” category is likely to be low and difficult to extract as a factor.

  In the present embodiment, the expression (2) is taken as an example. However, the expression is not limited to the expression (2). The factor word has a high frequency of appearance of words in the category. It is only necessary to be based on the concept of a word whose appearance frequency decreases with increasing rank. For example, the calculation divided by df (D, t) may not be executed, and the log of log (m− | j + 1−i |) may be excluded.

  After completion of repetition of all words (repetition end of step S1103), list1 (Ci) in category Ci is sorted in the order of score (Ci, t), and the top K items (specified arbitrarily by the user, here 3) ) Is extracted and left in list1 (Ci) (step S1106). That is, the words other than the top K words are deleted.

  After the repetition for all Ci (repetition end of step S1102), the factor word extraction unit 41 outputs list1 to the complex factor word extraction unit 42 (step S1107). Thereby, the factor word extraction process is completed.

  FIG. 11 shows an example of list1 output in step S1107. As shown in FIG. 11, in the “extremely dangerous” category, “tire”, “drop off”, and “accelerator” are extracted as factor words in list1. In the “danger” category, “hose”, “hub”, and “handle” are extracted. In the “somewhat dangerous” category, “seat belt” and “indicator light” are extracted. In the “safety” category, “cooling”, “speaker”, and “not effective” are extracted. As described above, a single word is extracted as a factor in each ranked category.

  Returning to the description of FIG. When the factor word generation processing by the factor generation unit 41 ends in step S403, the compound factor word extraction unit 42 performs compound factor word extraction processing (step S404).

  With reference to FIG. 12 and FIG. 13, an example of the complex factor word extraction process executed by the complex factor word extracting unit 42 in step 404 of FIG. 5 will be described.

  In the compound factor word extraction process, when words are combined among the words extracted in step S403, a combination of words that causes an increase / decrease in the category ranking is extracted. Further, for word sets that are not extracted by the factor word extraction unit 41, combinations of words that cause an increase or decrease in category ranking are extracted when words are similarly combined.

  First, the complex factor word extraction unit 42 initializes list2, which is a list of extraction results in the complex factor word extraction process (step S1301). Then, it repeats about each category Ci with a ranking (step S1302), and performs the process of step S1303-step S1318.

  When one category is selected from each category with ranking in step S1302, the complex factor word extraction unit 42 sets the initial value of the variable conbCnt that counts word combinations to 2 and sets the factor word to the termSet representing the set of words to be combined. The output value of the extraction unit 41 (input value of the compound factor word extraction unit 42) is substituted for the initial value null of the variable termConb that holds a set of combined words (step S1303). Then, combCnt pieces are combined for each word of termSet (step S1304).

  Here, the words combined in step S1304 are limited to words whose number of documents in which the word appears in category Ci (hereinafter referred to as the number of appearing documents) is equal to or greater than a predetermined threshold. By doing so, the processing speed of the combination is improved.

  For example, it is assumed that the threshold of the number of appearing documents is 3, conbCnt = 2, and termSet is word A, word B, word C, and word D. When the number of appearing documents in the category Ci including the word A, the word B, the word C, and the word D is 3, 4, 2, and 4, respectively, the word C is less than a certain value. Combinations including the word C are not targeted, and “word A and word B”, “word A and word D”, and “word B and word D” are termComb.

  Similarly, when conbCnt = 3, termSet becomes “word A and word B”, “word A and word D”, “word B and word D”, and termConb becomes “word A, word B and word D”. Can be considered. Here, when the numbers of appearance documents in the category Ci including the respective words are 2, 3, and 4, respectively, “word A and word B” are not satisfied. Therefore, a combination including “word A and word B”, that is, “word A, word B, and word D” is not a combination target. Such a combination idea is called an a priori algorithm.

  If the termComb extracted in step S1304 is not null (step S1305) due to the a priori algorithm concept, the termComb is repeated for each word t (step S1306), and the number of appearing documents in Ci including the word t is statistically significant. It is determined whether it is larger (step S1307). The determination of whether it is significantly large is the same as the method of step S1104 in FIG.

  When the number of appearing documents in Ci including the word t is statistically significantly large (Yes in step S1307), the compound factor word extraction unit 42 obtains score (Ci, t), and substitutes t and its score for list2. (Step S1308).

  When the number of appearance documents in Ci including the word t is not statistically significantly large (No in step S1307), the process returns to step S1306, and the process is repeated with another word t. The calculation of score (Ci, t) here is performed by the same method as S1105 in FIG.

  After the repetition of all words (repetition end of step S1306), the complex factor word extraction unit 42 increments conbCnt by one and substitutes termConb into termSet (step S1309). Thereafter, the compound factor word extraction unit 42 repeats the processing of steps S1304 to S1309 until it is determined that termConb is null in S1305.

  After termConb is determined to be null (No in step S1305), the complex factor word extraction unit 42 sorts list2 (Ci) in the category Ci in the order of score (Ci, t), and sets the top K word pairs ( The extracted word pairs are put in list2 (Ci), and only the unextracted words are left in list (Ci) as a single factor word (step S1310). That is, when a combination of words is extracted by the complex factor word extraction process, these words are deleted from the output list 1 as a result of the factor word extraction process. If the top K word pairs are not extracted, they are put in all the extracted word pairs list2 (Ci).

  Through the processes in steps S1301 to S1310 described above, a word that causes an increase / decrease in the category rank is extracted when a single word extracted by the factor word extraction unit 41 is combined. Processing in which the processing in subsequent steps S1311 to S1318 extracts a combination of words that cause a rise or fall in the category ranking when words that have not been extracted by the factor word extraction unit 41 are combined. It is.

  First, the complex factor word extraction unit 42 sets the initial value of the variable conbCnt for counting word combinations to 2, the output list3 to null, and the termSet representing the set of words to be combined, among all the words of Ci, the factor word Null is substituted for the initial value of a variable termConb that holds a set of words that are a combination of words that have not been output by the extraction unit 41, that is, words that have not been input to the complex factor word extraction unit 42 (step S1311).

  In subsequent steps S1312-S1316, the same method as S1304-S1308 is performed.

  That is, the compound factor word extraction unit 42 combines conbCnt words with each word of the termSet according to the concept of the a priori algorithm (step S1312). If there is a value for the termComb extracted by the a priori algorithm concept (Yes in step S1313), the process is repeated for each word t of the termConb (step S1314), and the number of documents appearing in Ci including the word t is statistically significant. (Step S1315).

  When the number of appearance documents in Ci including the word t is statistically significantly large (Yes in step S1315), the compound factor word extraction unit 42 obtains score (Ci, t), and substitutes t and its score into list3. (Step S1316).

  When the number of documents appearing in Ci including the word t is not statistically significant (No in step S1315), the process returns to step S1314 and is repeated with another word.

  After repetition of all words in each word t of termConb (repetition end of step S1314), the compound factor word extraction unit 42 increments conbCnt by one and substitutes termConb into termSet (step S1317). The factor word extraction unit 42 repeats the processing in steps S1312 to S1317 until termConb becomes null in S1313.

  When termConb is null (No in step S1313), the compound factor word extraction unit 42 sorts list3 (Ci), extracts the top K (two here), and puts it in list2 (Ci). After the repetition of all the categories Ci (repeated end of step S1302), the compound factor word extraction unit 42 outputs list2 (step S1319). Thereby, the complex factor word extraction process is completed.

  FIG. 14 shows an example of list2 output by the complex factor word extraction unit 42 in step S1319. As illustrated in FIG. 14, in the “extremely dangerous” category, a word that combines “tire” and “dropped” is extracted from list2. In the “danger” category, words combining “handle” and “heavy” are extracted. In the “slightly dangerous” category, words combining “indicator light” and “not applicable” are extracted. In the “safety” category, words combining “cooling” and “not effective” are extracted.

  By presenting the combination of words, which are these complex factors, to the user, the user can easily understand the ranking factor.

  In the list 3 of FIG. 14, in the “extremely dangerous” category, words combining “fuel” and “leakage” and words combining “brake” and “not effective” are extracted. In the “danger” category, words that combine “fuel” and “remaining amount” and words that combine “brake” and “hard” are extracted. In the “somewhat dangerous” category, words combining “door”, “lock”, and “cannot” are extracted. In the “safety” category, words combining “door” and “not open” are extracted.

  As shown in list 3, when a factor word is extracted from a set of words not extracted by the factor word extraction process, “extremely dangerous” and “dangerous” like “fuel” and “brake” are extracted. Words that appear in both categories may be extracted. In other words, words such as “fuel” and “brake” are not factors for each category, but by combining “leakage”, “remaining amount”, “not effective” and “not effective” It is possible to extract combinations of words that cause “danger” and “danger”.

  Returning to the description of FIG. When the complex factor word extraction process is completed, the factor categorization cebu 43 of the factor generation unit 4 generates a new category from the extracted words and word combinations as a result of the factor extraction process in steps S403 and S404. A category generation process is performed (step S405).

  The factor category generation processing by the factor category generation unit 43 will be described with reference to FIGS.

  FIG. 15 is a flowchart illustrating an example of a factor category generation process executed by the factor category generation unit 43. In the factor category generation process, the factor category generation unit 43 generates a new category based on the words extracted in steps S403 and S404 in each ranked category Ci, and the generated new category belongs to the ranked category. This is a process for classifying document data.

  The factor category generating unit 43 repeats for each ranked category Ci (step S1401), and each word and word of the category stored in the list1 and list2 output by the factor word extracting unit 41 and the compound factor word extracting unit 42 are stored. It repeats about a combination (step S1402), collects the document data containing the said word and the combination of a word, and produces | generates a category (step S1403). The category name generated at this time is a word or a combination of words from which document data is collected.

  After repeating for all categories Ci (repeated end of step S1401), the generated category is registered in the classification result storage unit 3 (step S1404). Thereby, the factor category generation process ends.

  After the factor category generation process, the newly generated category is displayed on the user interface unit 3.

  FIG. 16 shows an example of a category screen displayed on the user interface unit 3 after the factor category generation process. The category display screen 51 shown in FIG. 16 is a document that includes the words and word combinations extracted in steps S403 and S404 immediately under the categories of “extremely dangerous”, “dangerous”, “somewhat dangerous”, and “safety”. Categories 901 to 915 are generated.

  Next, step S406 in FIG. 5 will be described. The biaxial map visualization unit 32 creates and displays a biaxial map with the categories generated in step S405 and the set of categories with ranks input in step S401 as axes (step S406).

  Here, the biaxial map visualization process will be described with reference to FIGS. 17 and 18.

  The biaxial map visualization unit 32 initializes viewList, which is a list for storing data to be displayed on the biaxial map (step S1501). Next, the biaxial map visualization unit 32 repeats for each ranked category Ci (step S1502), repeats for the new category Cf generated by the factor category generation unit 43 (step S1503), and sets both Ci and Cf. Appearing document data is acquired and stored in viewList in association with the number of acquired document data, the category name of the ranking category Ci, and the category name of the new category Cf (step S1504).

  After repeating for all new categories Cf (repeated end of step S1503), repeated for other ranked categories and repeated for all ranked categories Ci (repeated end of step S1502), the biaxial map visualization unit 32 The viewList is output (step S1505). Thereby, the biaxial map visualization process ends.

  FIG. 18 shows an example of a biaxial map screen that displays the viewList output by the biaxial map visualization unit 32. As shown in FIG. 18, in the two-axis map screen 60, the new category generated by the factor category generation unit 43 is the vertical axis, the ranked category input to the factor word extraction unit 41 is the horizontal axis, and the vertical axis In the column where the horizontal axis and the horizontal axis cross, the number of document data including the category names of both the vertical axis and the horizontal axis is shown. At this time, for a category that is a word and a combination of words extracted in the factor extraction process in each category with ranking, a column for displaying the number of document data is highlighted. Here, diagonal lines are displayed.

  As described above, according to the document analysis apparatus 10 of the present embodiment, words that cause an increase / decrease in rank are extracted based on the rank relation of categories for categories with ranks set in advance. Thus, the user can grasp the factor of the ranking assigned to the category. As a result, the user can easily grasp the contents of the document data.

  Further, according to the document analysis apparatus 10 of the present embodiment, not only extraction of a single word as a factor, but also extraction can be performed even when factors are complex, that is, when words are combined. At this time, the document analysis apparatus 10 can also extract a combination of words that can be a factor when combined even if a single word is not a factor. Thereby, the user can easily grasp the contents of the document data.

  Further, the document analysis apparatus 10 according to the present embodiment uses the category generated by the factor category generation unit 43 as the classification axis 1, each category with a preset rank as the classification axis 2, and the category of the classification item of the classification axis 1. Then, cross tabulation is performed by obtaining the number of document data classified into both of the category items of the classification axis 2 and all the combinations of the plurality of categories to create a two-axis map. Thereby, the user can easily grasp the contents of the document data.

  As another example of the analysis by the document analysis apparatus 10 of the present embodiment, there is a case where product information is desired to be analyzed. When the product information is classified into the sales category for each age group, it is possible to extract a sales factor for a predetermined age group.

  As mentioned above, although embodiment of this invention was described, this embodiment is shown as an example and is not intending limiting the range of invention. The novel embodiment can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. This embodiment and its modifications are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

  For example, when the user selects a predetermined category, the document analysis apparatus 10 of the present embodiment extracts factor words for all ranked categories that are child categories belonging to the category, but the user selects the ranked category. It is also possible to extract factor words for only the category with the ranking.

DESCRIPTION OF SYMBOLS 1 ... Document storage part, 2 ... Classification result storage part, 3 ... User interface part, 31 ... Category display operation part, 32 ... Biaxial map visualization part, 4 ... Factor generation part, 41 ... Factor word extraction part, 42 ... Compound Factor word extraction unit, 43 ... Factor category generation unit

Claims (6)

A document analysis apparatus comprising a document storage unit for storing document data and analyzing the document data,
A plurality of categories with ranking for classifying the document data, and a classification result storage unit for storing a hierarchical structure of the categories ;
In one category of the plurality of categories, from the word group included in the document data classified into the category, the appearance frequency of the word in another category in which the appearance frequency of the word in the category is in the same hierarchy as the category A factor word extraction unit that extracts words that are higher in appearance frequency and decrease in frequency as the rank of the appearance frequency of the word in the other category becomes farther from the category;
A document analysis apparatus comprising:   The factor word extraction unit creates a combination of extracted words, and in one category of the plurality of categories, the appearance frequency in the category is higher than the appearance frequency in the other category from the created word combination. The document analysis apparatus according to claim 1, wherein a combination of the words whose appearance frequency in the other category decreases and decreases as the rank becomes far from the category is extracted.   The factor word extraction unit creates a combination of words included in the document data classified into the category other than the word of the extraction result, and in one category of the plurality of categories from the combination of words 2. The combination of the words whose frequency of occurrence in the category is higher than the frequency of occurrence in the other category and whose frequency of occurrence in the other category decreases as the rank becomes farther from the category is extracted. 3. The document analysis apparatus according to any one of items 2.   A factor category generating unit that generates a category in a lower hierarchy of a category to be extracted by the factor word extracting unit for classifying document data including the word or a combination of words extracted by the factor word extracting unit. The document analysis apparatus according to any one of claims 1 to 3. The number of document data classified on both axes with the category generated by the factor category generation unit as the first classification axis and the classification category with ranking stored in the classification result storage unit as the second classification axis document analysis device according to any one of claims 1 to 4 comprising a two-axis map creating unit that displays. A document analysis program of a document analysis apparatus comprising a document storage unit for storing document data and analyzing the document data,
In the computer of the document analysis device,
A plurality of categories with ranking for classifying the document data, and a function of storing a hierarchical structure of the categories;
In one category of the plurality of categories, from the word group included in the document data classified into the category, the appearance frequency of the word in another category in which the appearance frequency of the word in the category is in the same hierarchy as the category A function of extracting words that are more than the appearance frequency and that the appearance frequency of the word in the other category decreases as the rank becomes farther from the category;
Document analysis program that realizes.

Images