JP2014056516A - Device, method and program for extracting knowledge structure out of document set - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a system in which on the basis of a relationship between extraction keywords in a document set, extraction keywords having a close relationship can be disposed close to each other in a flexible manner.SOLUTION: A contribution degree for discriminating from other documents is calculated for a keyword contained in a document having been subjected to field classification; the document is classified into a unit on the basis of the calculated contribution degree by using a self-organization map, and displayed; and thereafter, arrangement information of the keyword is calculated from an appearance frequency of the keyword contained in the document in the unit, and displayed in accordance with the unit.

Description

  The present invention relates to an information processing apparatus and control for extracting a knowledge structure from an electronic document set.

  In recent years, an increasing number of companies are working on the use of big data. By analyzing the enormous amount of data that has been stored as records, it is an effort to discover hidden knowledge and use it for corporate activities.

  Currently, data mining that analyzes structured data such as sales data is the mainstream, but text mining that analyzes unstructured data such as business documents using natural language processing technology is also the mainstream. Attention is increasing.

  Text mining is a simple function for summarizing keywords and co-occurrence frequencies in a document or sentence and displaying them in a table or graph based on the search conditions entered by the user, or to express an outline of a document set. There is a function like a keyword map that shows characteristic keywords and their relationships in a single figure.

  For example, the relationship between related terms including the search term is displayed on the map, the words that are highly relevant to the search term are arranged in the vicinity, and the font size of the high words on the map is shown to be large. A technique for informing the user of the fact that the strength is strong at a glance is disclosed (for example, see Patent Document 1).

  In addition, when performing a document search, classification is performed using the appearance frequency of the word included in the entire document, not limited to the appearance frequency of the word included in the document having the search keyword. A technique for searching for a balanced document by specifying a simple word and using the co-occurrence strength between characteristic words and displaying the relationship between the features in a graph format is disclosed. (For example, refer to Patent Document 2).

JP 2008-250625 A JP-A-10-74210

  However, in the system of Patent Document 1, the extracted keywords are arranged depending on the degree of association with the search keyword, and the relationship between the extracted keywords cannot be expressed.

  Moreover, in the system of the above-mentioned patent document 2, since the link generated based on the co-occurrence relationship of the extracted keywords is generated, the relationship between the extracted keywords can be expressed.

  However, when one extracted keyword has only one link to the parent as in the embodiment, there is a possibility that the keyword at the end of the link is connected to a different link even if the relationship is deep. In addition, when two or more links are possible, it is considered that realistic links and arrangement are difficult.

  Accordingly, an object of the present invention is to provide a mechanism capable of flexibly arranging closely related extracted keywords based on the relationship between extracted keywords in a document set.

  The first invention for achieving the above object is a field classification means for classifying the document based on the document content, and a contribution for discriminating the keyword included in the document from other documents. A unit for classifying the document into units using a self-organizing map based on the contribution calculated by the contribution degree calculating unit, and the unit classification unit Included in the document in the unit, display means for displaying the units, cluster generation means for generating a cluster of each unit using hierarchical clustering based on each unit classified by the unit classification means The arrangement information of the keyword with respect to the display means from the unit specified from the appearance frequency of the keyword A link information is generated from the association between the cluster to which the specified unit belongs and another cluster by obtaining the degree of coincidence between the specified unit and the cluster, and the display information is calculated on the display means. A knowledge structure extraction device comprising: a link generation display means for displaying.

  A second invention for achieving the above object is a knowledge structure extraction method for extracting and displaying a related keyword from a document in a knowledge structure extraction device, wherein the field classification means of the knowledge structure extraction device includes a document content A field classification step of classifying the document on the basis of the document, and a contribution calculation means of the knowledge structure extraction device for calculating a contribution for discriminating from another document with respect to a keyword included in the document And a unit classification step of classifying the document into units using a self-organizing map based on the contribution calculated in the contribution calculation step. The display unit of the knowledge structure extraction device includes a display step for displaying the units classified by the unit classification step, and the knowledge The cluster generation means of the structure extraction device includes a cluster generation step of generating a cluster of each unit using hierarchical clustering based on each unit classified by the unit classification step, and an arrangement of the knowledge structure extraction device An information calculation means includes an arrangement information calculation step for calculating the arrangement information of the keyword with respect to the display step from a unit specified from the appearance frequency of the keyword included in the document in the unit, and a link generation means of the knowledge structure extraction device Performing a link generation display step of generating and displaying a link from the association between the cluster to which the identified unit belongs and another cluster by obtaining a degree of match between the identified unit and the cluster. Knowledge structure extraction method characterized by .

  A third invention for achieving the above object is a program executed in a knowledge structure extraction device that extracts and displays related keywords from a document, and the knowledge structure extraction device is based on the contents of the document. Field classification means for classifying a document, contribution calculation means for calculating a contribution for discriminating a keyword included in the document from other documents, and contribution calculated by the contribution calculation means Unit classification means for classifying the document into units using a self-organizing map, display means for displaying the units classified by the unit classification means, and each unit classified by the unit classification means Cluster generating means for generating a cluster of each unit using hierarchical clustering based on An arrangement information calculation means for calculating the arrangement information of the keyword with respect to the display means from a unit specified from the appearance frequency of the keyword included in the document in a document, and a degree of coincidence between the specified unit and the cluster Thus, the program is characterized by functioning as link generation display means for generating a link from the association between the cluster to which the identified unit belongs and another cluster and displaying the link on the display means.

  According to the present invention, it is possible to extract characteristic keywords from a specified document set and to place extracted keywords that are closely related to each other based on the relationship between the extracted keywords, which is more intuitive and easy to understand. A figure can be created.

It is a system configuration figure showing an example of the composition of the knowledge structure extraction system of the present invention. It is a block diagram which shows the hardware constitutions of the information structure apparatus applicable to the knowledge structure extraction apparatus of this invention, and a user terminal. It is a flowchart which shows an example of the knowledge structure extraction process in the knowledge structure extraction apparatus in this invention. It is a flowchart which shows an example of the document collection process in the knowledge structure extraction process of this invention. It is a figure which shows an example of the URL log | history acquired from the utilization terminal in this invention. It is a flowchart which shows an example of the field analysis process in the knowledge structure extraction process of this invention. It is a figure which shows the outline | summary of the weighted field information acquisition process in the field analysis process of this invention. It is a flowchart which shows an example of the keyword extraction process in the knowledge structure extraction process of this invention. It is a figure which shows an example of the document information table in the document information storage area | region in this invention. It is a flowchart which shows an example of the document arrangement | positioning process in the knowledge structure extraction process of this invention. It is a figure which shows an example of the self-organization map used in the document arrangement | positioning process of this invention, and a display position. It is a figure which shows an example of the unit information of this invention. It is a figure which shows the outline | summary of the clustering result of the self-organizing map unit in the document arrangement | positioning process of this invention. It is a figure which shows an example of the clustering table of the self-organizing map unit produced | generated in the document arrangement | positioning process of this invention. It is a figure which shows an example of the document information table in the document information storage area | region in this invention. It is a flowchart which shows an example of the keyword arrangement | positioning process in the knowledge structure extraction process of this invention. It is a figure which shows an example of the keyword arrangement | positioning table produced | generated in the keyword arrangement | positioning process of this invention. It is a figure which shows an example of a mode that the keyword was arrange | positioned on the self-organization map based on the keyword arrangement | positioning information produced | generated in the keyword arrangement | positioning process of this invention. It is a flowchart which shows an example of the link production | generation process in the knowledge structure extraction process of this invention. It is a figure which shows an example of a mode that the link was provided to the keyword arrange | positioned on the self-organization map based on the link information produced | generated in the link production | generation process of this invention. It is a figure which shows the outline | summary of the weighted field information acquisition process in the field analysis process of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a system configuration diagram showing an example of the configuration of the knowledge structure extraction system of the present invention.

  FIG. 1 shows a configuration in which a knowledge structure extraction apparatus 100 and one or a plurality of user terminals 130 are connected via a local area network. In addition, the user terminal 130 can be connected to the external network 140.

  The knowledge structure extraction device 100 acquires / analyzes the contents of the web page from the browsing history of the web page acquired from the user terminal 130, creates a diagram showing an outline of the browsing content of the web page, and returns it to the user terminal 130. .

  The user terminal 130 sends the browsing history of the web pages browsed via the external network 140 to the knowledge structure extracting device 100, and obtains an overview of browsing contents generated by the knowledge structure extracting device 100.

  Further, the document acquired from the knowledge structure extraction system and the user terminal 130 of the present embodiment may be other than the web browsing history.

  Hereinafter, the hardware configuration of the information processing apparatus applicable to the knowledge structure extraction apparatus 100 and the user terminal 130 illustrated in FIG. 1 will be described with reference to FIG.

  FIG. 2 is a block diagram illustrating a hardware configuration of an information processing apparatus applicable to the knowledge structure extraction apparatus 100 and the user terminal 130 illustrated in FIG.

  In FIG. 2, reference numeral 201 denotes a CPU that comprehensively controls each device and controller connected to the system bus 204. Further, the ROM 202 or the external memory 211 is necessary to realize a BIOS (Basic Input / Output System) or an operating system program (hereinafter referred to as an OS), which is a control program of the CPU 201, or a function executed by each server or each PC. Various programs to be described later are stored.

  A RAM 203 functions as a main memory, work area, and the like for the CPU 201. The CPU 201 implements various operations by loading a program or the like necessary for execution of processing from the ROM 202 or the external memory 211 into the RAM 203 and executing the loaded program.

  An input controller 205 controls input from a keyboard (KB) 209 or a pointing device such as a mouse (not shown). A video controller 206 controls display on a display device such as a CRT display (CRT) 210. In FIG. 2, although described as CRT 210, the display device is not limited to the CRT, but may be another display device such as a liquid crystal display. These are used by the administrator as needed.

  A memory controller 207 is provided in an external storage device (hard disk (HD)), flexible disk (FD), or PCMCIA card slot for storing a boot program, various applications, font data, user files, editing files, various data, and the like. Controls access to an external memory 211 such as a compact flash (registered trademark) memory connected via an adapter.

  A communication I / F controller 208 connects and communicates with an external device via a network (for example, the LAN 400 shown in FIG. 1), and executes communication control processing in the network. For example, communication using TCP / IP is possible.

  Note that the CPU 201 enables display on the CRT 210 by executing outline font rasterization processing on a display information area in the RAM 203, for example. In addition, the CPU 201 enables a user instruction with a mouse cursor (not shown) on the CRT 210.

  Various programs to be described later for realizing the present invention are recorded in the external memory 211 and executed by the CPU 201 by being loaded into the RAM 203 as necessary. Furthermore, definition files and various information tables used when executing the program are also stored in the external memory 211, and a detailed description thereof will be described later.

  The overall flow of the knowledge structure extraction system in this embodiment will be described below.

  The user terminal 130 transmits all or part of the browsing history of the web page in a specific period to the knowledge structure extraction device 100 according to a user instruction. You may make it the structure which a user selects the browsing history of the web page to transmit.

  When receiving the web page browsing history from the user terminal 130, the knowledge structure extracting device 100 analyzes the content of the web page and returns the extracted knowledge structure in a format that can be displayed on the user terminal 130.

  When the user terminal 130 receives the knowledge structure extracted from the knowledge structure extraction apparatus 100, the user terminal 130 displays the received knowledge structure in the browser 121.

  In the present embodiment, a self-organizing map is used for classification of web pages and arrangement on a two-dimensional plane. In addition, the Ward method is used for clustering the units classified by the self-organizing map.

References
(1) T. Kohonen, “The self-organizing map”, Proceeding of IEEE, vol. 78, no. 9, Sept. 1990
(2) Joe H. Ward, Jr., Hierarchical Grouping to Optimize an Objective Function, Journal of the American Statistical Association, Vol. 58, 1963

  Hereinafter, with reference to FIG. 3, a knowledge structure extraction process in the knowledge structure extraction system of the present embodiment will be described.

  In step S <b> 301, the document collection unit 101 selects a web page as a knowledge structure extraction target from the browsing history of the web page received from the user terminal 130, acquires the body text of the web page, and stores it in the document information storage area 102. save.

  In step S <b> 302, the field analysis unit 103 analyzes the body text of the web page stored in the document information storage area 102 and extracts field information 104.

  In step S <b> 303, the keyword extraction unit 105 analyzes the body text of the web page stored in the document information storage area 102 and extracts the keyword information 106.

  Steps S302 and S303 may be configured to execute processes in parallel, or may be configured to extract both the field information 104 and the keyword information 106 in the same process.

  In step S304, the document placement unit 107 uses the self-organizing map for the field information 104 to determine a position where a web page with similar contents is close to the two-dimensional plane for each web page, and the document placement. Information 108 is generated.

  In step S305, the keyword arrangement unit 109 refers to the document arrangement information 108 generated in step S304 for the keyword of the keyword information 106 extracted in step S303, determines the position of the keyword on the two-dimensional plane, Generated as keyword arrangement information 110. Further, a cluster highly related to the keyword is identified from the information of the document to which the keyword belongs and the unit information of the self-organizing map to which the document belongs.

  In step S306, the link generation unit 111 generates link information between keywords according to the hierarchical relationship of the cluster information associated in step S305. Various information generated including link information is stored as knowledge structure information in the knowledge structure information storage area 112.

  In step S307, the display / editing unit 113 generates a diagram summarizing the entire document set as shown in FIG. 20 from the knowledge structure information stored in the knowledge structure information storage area 112, and transmits the generated diagram to the user terminal. .

  FIG. 4 describes details of the document collection processing S301.

  In step S401, the document collection unit 101 receives a web page browsing history from the user terminal as a URL list. FIG. 5 shows an example of a URL list.

  In step S402, the document collection unit 101 starts repetitive processing up to step S407 on the received URL list.

  In step S403, the document collection unit 101 determines whether the URL being processed is a processing target. If the URL is a processing target, the process proceeds to step S404. If the URL is not a processing target, the process proceeds to step S407.

  The determination as to whether or not it is a target of processing is arbitrary, for example, whether the browsing time is within the last day, only a specific user is targeted, or a specific URL that is periodically browsed is excluded. It may be based on conditions.

  In step S404, the document collection unit 101 acquires the document indicated by the URL being processed via the network.

  In step S405, the document collection unit 101 extracts body text from the acquired document. You may perform the process which removes unnecessary parts, such as an advertisement.

  In step S406, the document collection unit 101 stores the body text in the document information table 901 in the document information storage area 102 together with the URL information. FIG. 9 shows an example of the document information table 901.

  In step S407, the document collection unit 101 performs the processing from step S402 when there is a next URL. If there is no next URL, the process ends.

  A specific example of the document collection process S301 will be described.

  In step S401, the document collection unit 101 receives the web page browsing history shown in FIG. 5 as a URL list from the user terminal.

  In step S402, the document collection unit 101 starts the iterative process up to step S407 for the URL 501.

  In step S403, the document collection unit 101 determines whether the URL 501 is a processing target. Here, the URL whose accessed user is “nagai” is the processing target. Since the user of the URL 501 is “sakai”, the process proceeds to step S407.

  In step S407, the document collection unit 101 moves to step S402 because there is the next URL 502.

  Thereafter, the document collection unit 101 performs the same processing for the URLs after the URL 502 until just before the URL 503.

  In step S402, the document collection unit 101 starts the iterative process up to step S407 for the URL 503.

  In step S403, the document collection unit 101 moves the process to step S404 because the user of the URL 503 is “nagai”.

  In step S404, the document collection unit 101 acquires the document indicated by the URL 503 via the network.

  In step S405, the document collection unit 101 extracts body text from the acquired document. You may perform the process which removes unnecessary parts, such as an advertisement.

  In step S406, the document collection unit 101 stores the extracted text text and URL information as document information 907 in the document information table 901 shown in FIG. At this time, the field column 905 and the keyword column 906 of the document information 907 are empty.

  In step S407, the document collection unit 101 performs the processing from step S402 because there is the next URL 504.

  Thereafter, the same processing is repeated.

  FIG. 6 describes details of the field analysis processing S302.

  In step S <b> 601, the field analysis unit 103 starts repetitive processing up to step S <b> 604 on the document information stored in the document information table 901 in the document information storage area 102.

  In step S602, the field analysis unit 103 analyzes the text of the document information and specifies the field to which the document belongs. In the field classification of the present invention, the field to which the document belongs is not determined as one, but is expressed as a set of a plurality of specified fields and degrees belonging to the field as shown in FIG. Hereinafter, a set of a plurality of fields and degrees for a document is referred to as weighted field information. Document field classification can be implemented using existing methods such as naive Bayes classifiers.

  In the naive Bayes classifier, the probability that a document belongs to the document group can be obtained from the frequency of occurrence of the word constituting the document in the document group belonging to a certain field. Based on this probability, the degree belonging to the field is quantified and acquired as weighted field information. The weighted field information may be acquired by a method other than naive Bayes.

  In step S603, the field analysis unit 103 adds the weighted field information acquired in step S602 to the document information table 901 in the document information storage area 102.

  In step S604, the field analysis unit 103 performs the processing from step S601 when there is next document information. If there is no next document information, the process ends.

  A specific example of the field analysis process S302 will be described.

  In step S601, the field analysis unit 103 starts repetitive processing up to step S604 for the document information 907 stored in the document information table 901.

  In step S602, the field analysis unit 103 analyzes the text 701 in the text column 904 of the document information 907, and obtains weighted field information 702.

  In step S603, the field analysis unit 103 adds the weighted field information 702 acquired in step S602 to the field column 905 of the document information 907 in the document information table 901.

  In step S604, the field analysis unit 103 performs the processing from step S601 because there is the next document information 908.

  Thereafter, the same processing is repeated.

  FIG. 8 describes details of the keyword extraction process S303.

  In step S801, the keyword extraction unit 105 starts the iterative process up to step S604 on the document information stored in the document information table 901 in the document information storage area 102.

  In step S802, the keyword extraction unit 105 analyzes the text of the document information and extracts keywords included in the document.

  In the keyword extraction of the present invention, the degree of contribution to discriminating from other documents is quantified for each keyword constituting the document, and those having a high degree of contribution to discrimination are extracted as keywords together with the degree value. Hereinafter, a set of the degree of contribution to the keyword and the discrimination is referred to as weighted keyword information.

  The degree of contribution to keyword document discrimination can be determined using tf / idf values. A value other than the tf / idf value may be used as the degree of contribution of the keyword to document discrimination. In order to obtain the tf / idf value, statistical information such as the frequency of occurrence of keywords in the document set is required.

  In many cases, the entire document to be analyzed is used as a document set. In the present invention, statistical information extracted in advance from a large-scale document set including various documents is used. This is to express the entire analysis target and prevent keywords that frequently appear in the entire analysis target from being excluded from the extraction target.

  In step S <b> 803, the keyword extraction unit 105 adds the weighted keyword information acquired in step S <b> 802 to the document information table 901 in the document information storage area 102.

  In step S804, the keyword extraction unit 105 performs the processing from step S801 when there is next document information. If there is no next document information, the process ends.

  A specific example of the keyword extraction process S303 will be described.

  In step S <b> 801, the keyword extraction unit 105 starts the iterative processing up to step S <b> 604 for the document information 907 stored in the document information table 901.

  In step S802, the keyword extraction unit 105 analyzes the text 701 in the text column 904 of the document information 907, and acquires weighted keyword information 2101 (see FIG. 21).

  In step S803, the keyword extraction unit 105 adds the weighted keyword information 2101 acquired in step S802 to the keyword column 906 of the document information 907 in the document information table 901.

  In step S804, the keyword extraction unit 105 performs the processing from step S801 because there is the next document information 908.

  Thereafter, the same processing is repeated.

  FIG. 10 describes details of the document arrangement processing S304.

  In the document placement process of this embodiment, the weighted field information is regarded as a vector, and the document is placed on a two-dimensional plane by applying a self-organizing map.

In step S <b> 1001, the document placement unit 107 learns a self-organizing map for the document information stored in the document information table 901 in the document information storage area 102. FIG. 24 shows an example of a formula for calculating the distance between weighted fields (vectors) necessary for learning and classification of the self-organizing map and clustering by the Ward method. The distance may be defined by other methods.
"Formula 1"

  FIG. 11 shows an example of the self-organizing map visualized. The hexagon in the figure is called a unit, and the self-organizing map can be classified into any unit when learning is completed. The numbers in the units in FIG. 11 are unit identification numbers, and information on each unit is stored in the unit information table shown in FIG.

  Each unit has field information in a weighted field column 1207. The y column 1203 means the vertical position of the unit starting from the upper left, and the x column 1204 means the horizontal position of the unit starting from the upper left.

  In step S <b> 1002, the document placement unit 107 performs hierarchical clustering such as the Ward method on the self-organizing map unit for which learning has been completed. By hierarchical clustering, each unit is grouped in order of weighted field information.

  FIG. 13 shows an example of the result of hierarchical class ring. The number at the very end is the unit identification number, and the middle number is the cluster identification number.

  In step S1003, the document placement unit 107 stores cluster information.

  FIG. 14 shows an example of a table for storing cluster information.

  In step S1004, the document placement unit 107 starts the iterative process up to step S1007 for the document stored in the document information table 901 in the document information storage area 102.

  In step S1005, the document placement unit 107 identifies the unit to which the document information being processed belongs. The document being processed is determined to belong to the unit having the weighted field information closest to the extracted weighted field information.

  In step S <b> 1006, the document placement unit 107 stores unit information in the document information table 901 in the document information storage area 102.

  FIG. 15 shows an example of a document information table 901 including unit information.

  In step S1007, the document placement unit 107 performs the processing from step S1004 when there is next document information. If there is no next document information, the process ends.

  A specific example of the document arrangement process S304 will be described.

  In step S1001, the document placement unit 107 learns a self-organizing map for the document information stored in the document information table 901, and obtains a unit information table 1201.

  In step S1002, the document placement unit 107 performs hierarchical clustering on each unit included in the unit information table 1201 based on the mutual distance obtained from the field column 1207, and obtains the clustering result shown in the tree diagram of FIG. obtain.

  In step S <b> 1003, the document placement unit 107 stores the clustering result shown in the tree diagram of FIG. 13 to obtain the cluster information table 1401.

  In step S1004, the document placement unit 107 starts the iterative process up to step S1012 for the document information 907.

  In step S1005, the document placement unit 107 calculates the distance between the weighted field information of the document information 907 and the field column 1207 of each unit in the unit information table 1201, and the unit information 1208 having the smallest distance is the unit to which the document information 907 belongs. As specified.

  In step S1006, the document placement unit 107 adds the identification number of the unit information 1411 to the uid column 1501 of the document information 907 in the document information table 901 as shown in FIG.

  In step S1007, the document placement unit 107 performs the processing from step S1004 because the next document information 908 exists.

  Thereafter, by repeating the same processing, the arrangement positions of all documents on the self-organizing map are determined.

  FIG. 16 describes details of the keyword arrangement process S305.

  In step S1601, the keyword placement unit 109 starts the iterative process up to step S1607 for the document information in the document information table 901 in the document information storage area 102.

  In step S1602, the keyword placement unit 109 starts the iterative process up to step S1606 for the keyword included in the document information acquired in step S1601.

  In step S1603, the keyword placement unit 109 determines whether the keyword being processed is registered in the keyword placement information table 1701. If the keyword has already been registered in the keyword arrangement information table 1701, the process proceeds to step S1605. If no keyword is registered in the keyword arrangement information table 1701, the process proceeds to step S1604.

  In step S1604, the keyword placement unit 109 registers the keyword being processed in the keyword placement information table 1701.

  In step S1605, the keyword arrangement unit 109 adds the identification number of the unit to which the document information being processed belongs to the keyword being processed in the keyword arrangement information table 1701.

  If the unit identification number is already registered, the appearance frequency is increased by one.

  In step S1606, the keyword arrangement unit 109 performs the processing from step S1602 when there is a next keyword. If there is no next keyword, the process proceeds to step S1607.

  In step S1607, the keyword arrangement unit 109 performs the processing from step S1601 when there is next document information. If there is no next keyword, the process proceeds to step S1608.

  In step S1608, the keyword placement unit 109 starts the iterative process up to step S1613 for the keyword placement information in the keyword placement information table 1701.

In step S1609, the keyword placement unit 109 calculates a position from the unit information corresponding to the unit identification number of the keyword placement information being processed and the appearance frequency, and sets the position information as the keyword information being processed. An example of an expression for calculating the position is shown in Expression 2, but may be calculated by another method.
"Formula 2"

  In step S1610, the keyword placement unit 109 adds the keyword placement information calculated in step S1604 to the position information.

In step S <b> 1611, the keyword placement unit 109 refers to the cluster information acquired in step S <b> 1008 for the set of units including the keyword being processed, and acquires the cluster that most matches the set of units. The degree of match is determined by the equation shown in Equation 3.
"Formula 3"

  In step S1612, the keyword placement unit 109 adds the cluster information identification number acquired in step S1611 to the keyword placement information.

  In step S1613, when there is next keyword arrangement information, the keyword arrangement unit 109 performs processing from step S1608. If there is no next keyword arrangement information, the process ends.

  A specific example of the keyword arrangement process S305 will be described.

  FIG. 16 describes details of the keyword arrangement process S305.

  In step S1601, the keyword placement unit 109 starts the iterative process up to step S1607 for the document information 907 in the document information table 901.

  In step S1602, the keyword placement unit 109 starts the iterative process up to step S1606 for the keyword “high sensitivity” included in the document information 907.

  In step S1603, the keyword placement unit 109 moves the process to step S1604 because the keyword “high sensitivity” being processed is not registered in the keyword placement information table 1701.

  In step S <b> 1604, the keyword placement unit 109 registers the keyword “high sensitivity” being processed in the keyword placement information table 1701.

  In step S 1605, the keyword placement unit 109 adds the identification number “14” of the unit to which the document information 907 belongs to the keyword “high sensitivity” being processed in the keyword placement information table 1701.

  In step S <b> 1606, the keyword placement unit 109 performs the processing from step S <b> 1602 because there is the next keyword “camera”.

  Thereafter, the processes from step S1602 to S1606 are repeated, all the keywords of the document information 907 are processed, and the process proceeds to step S1607.

  In step S1607, the keyword placement unit 109 performs the processing from step S1601 because the next document information 908 is present.

  Thereafter, the processing from step S1601 to S1607 is repeated to obtain the keyword arrangement information table 1701.

  In step S1608, the keyword placement unit 109 starts the iterative process up to step S1613 for the keyword placement information 1709 in the keyword placement information table 1701.

In step S <b> 1609, the keyword placement unit 109 calculates a position from the unit information corresponding to the unit identification number of the keyword placement information 1709 and the appearance frequency. From the unit information table, the position of the unit with unit identification number 7 is (1, 2). Similarly, the positions of the other unit identification numbers are acquired, and the keyword positions (3.15, 2.55) are calculated from the expression (2).
y = (1 * 2 + 1 * 2 + 2 * 3 + 4 * 3 + 5 * 3 + 2 * 3 + 1 * 4 + 3 * 4 + 1 * 4) /20=3.15
x = (1 * 1 + 1 * 2 + 2 * 1 + 4 * 2 + 5 * 3 + 2 * 4 + 1 * 2 + 3 * 3 + 1 * 4) /20=2.55

  In step S1610, the keyword arrangement unit 109 adds the position information (3.15, 2.55) calculated in step S1604 to the y column 1704 and the x column 1705 of the keyword arrangement information 1709.

  In step S <b> 1611, the keyword placement unit 109 refers to the cluster information table 1401 for the unit set included in the keyword placement information 1709, and acquires the cluster that most matches the set of units.

  The cluster information 1402 is the highest level cluster including all units. The degree of coincidence between the unit set of the keyword arrangement information 1709 and the unit set of the cluster information 1402 is calculated from Equation 3.

The number of matching units is 9, {7, 8, 13, 14, 15, 16, 20, 21, 22}, and the number of units in the keyword arrangement information 1709 is also 9. The number of units of the cluster information 1402 is 36.

  (Matching degree with cluster information 1402) = (2 * 9) / (9 + 36) = 0.4

  The degree of coincidence between the unit set of the keyword arrangement information 1709 and the unit set of the cluster information 1403 is calculated from Equation 3. The number of matching units is 7 of {7, 8, 14, 15, 16, 21, 22}, and the number of units of the keyword arrangement information 1709 is {7, 8, 13, 14, 15, 16, 20, 21 , 22}. The number of units of the cluster information 1407 is 7 of {7, 8, 14, 15, 16, 21, 22}.

  (Matching degree with cluster information 1407) = (2 * 7) / (7 + 9) = 0.875

  Such calculation is performed for all clusters, and the cluster with the highest degree of match is selected. The cluster information 1407 most closely matches the keyword arrangement information 1709.

  In step S <b> 1612, the keyword arrangement unit 109 adds the identification number 64 of the cluster information 1407 to the cluster identification number 1707 of the keyword arrangement information 1709.

  In step S <b> 1613, the keyword arrangement unit 109 performs the processing from step S <b> 1608 because there is next keyword arrangement information 1710.

  Thereafter, the same processing is repeated.

  FIG. 19 describes details of the link generation processing S306.

  In step S1901, the link generation unit 111 starts the iterative process up to step S1912 for the cluster information acquired in step S1008.

  In step S1902, the link generation unit 111 acquires a keyword associated with the cluster being processed from the keyword arrangement information.

  In step S1903, the link generation unit 111 starts the iterative process up to step S1910 for the keyword associated with the cluster being processed.

  In step S1904, the link generation unit 111 acquires a cluster higher than the cluster being processed.

  In step S1905, the link generation unit 111 acquires a keyword associated with the cluster acquired in step S1904 as an upper keyword.

  In step S1906, the link generation unit 111 determines whether there is a higher keyword. If there is a higher keyword, the process moves to step S1907. If there is no higher keyword, the process proceeds to step S1909.

  In step S1907, the link generation unit 111 selects an upper keyword most relevant to the keyword being processed. As a selection criterion, the ratio calculated from the position of the keyword arrangement information (y, x) or the ratio included in the higher keyword arrangement information unit set in the keyword arrangement information unit set being processed is used. be able to. Other selection criteria may be used.

  In step S1908, the link generation unit 111 sets the upper keyword in the link information of the keyword arrangement information.

  In step S1909, the link generation unit 111 determines whether the upper cluster acquired in step S1904 is the highest. If the upper cluster is the highest, the process proceeds to step S1910. If the upper cluster is not the highest cluster, the process proceeds to step S1904.

  In step S1910, if there is a next keyword associated with the cluster being processed, the link generation unit 111 performs the processing from step S1903. If there is no next keyword, the process proceeds to step S1911.

  In step S1911, the link generation unit 111 performs the processing from step S1901 when there is a next cluster. If there is no next cluster, the process ends.

  A specific example of the link generation process S306 will be described.

  In step S1901, the link generation unit 111 starts the iterative process up to step S1912 for the cluster information 1402.

  In step S1902, the link generation unit 111 acquires keyword arrangement information 1711 (“high image quality”) associated with the cluster information 1402 (cluster identification number 72).

  In step S1903, the link generation unit 111 starts the iterative process up to step S1910 for the keyword arrangement information 1711 associated with the cluster information 1402.

  In step S1904, the link generation unit 111 tries to acquire cluster information at a higher level than the cluster information 1402, but does not exist. The upper cluster information cannot be acquired.

  In step S1905, the link generation unit 111 cannot acquire higher-order cluster information in step S1904, and therefore cannot acquire higher-order keywords.

  In step S1906, the link generation unit 111 moves the process to step S1909 because there is no higher keyword.

  In step S1909, the link generation unit 111 moves the process to step S1910 because the upper cluster information 1402 acquired in step S1904 is the highest.

  In step S1910, the link generation unit 111 moves to step S1911 because there is no next keyword arrangement information associated with the cluster information 1402.

  In step S1911, the link generation unit 111 performs the processing from step S1901 because there is next cluster information 1403.

  In step S1901, the link generation unit 111 starts the iterative process up to step S1912 for the cluster information 1403.

  In step S1902, the link generation unit 111 acquires keyword arrangement information 1712 (“aspheric lens”) associated with the cluster information 1403 (cluster identification number 71).

  In step S1903, the link generation unit 111 starts the iterative process up to step S1910 for the keyword arrangement information 1712 associated with the cluster information 1403.

  In step S1904, the link generation unit 111 acquires the upper cluster information 1402 of the cluster information 1403.

  In step S1905, the link generation unit 111 acquires keyword arrangement information 1711 associated with (cluster identification number 72) in the upper cluster information 1402 in step S1904.

  In step S1906, the link generation unit 111 moves the process to step S1907 because there is upper keyword arrangement information.

  In step S1907, the link generation unit 111 selects the keyword arrangement information 1711 as the upper keyword arrangement information most relevant to the keyword arrangement information 1712.

  In step S 1908, the link generation unit 111 sets the upper keyword identification number “25” in the upper keyword column 1708 of the keyword arrangement information 1712.

  In step S1910, the link generation unit 111 moves to step S1911 because there is no next keyword arrangement information associated with the cluster information 1403.

  In step S1911, the link generation unit 111 performs the processing from step S1901 because there is next cluster information 1404.

  Thereafter, the same processing is repeated.

  By using the keyword arrangement information, a mind map diagram as shown in FIG. 20 can be created.

  As described above, according to the present invention, it is possible to extract characteristic keywords from a designated document set, and to place extracted keywords that are closely related to each other based on the relationship between the extracted keywords. It becomes possible to create a figure that is easy to do.

  Although the embodiment has been described in detail above, the present invention can take an embodiment as, for example, a method, a program, a storage medium, or the like. Specifically, the present invention is applied to a system composed of a plurality of devices. The present invention may be applied, or may be applied to an apparatus composed of one device. The program according to the present invention is a program that allows a computer to execute (read) each processing method, and the storage medium according to the present invention stores a program that allows the computer to execute each processing method.

  The program in the present invention may be a program for each processing method of each device.

  As described above, a recording medium that records a program that implements the functions of the above-described embodiments is supplied to a system or apparatus, and a computer (or CPU or MPU) of the system or apparatus stores the program stored in the recording medium. It goes without saying that the object of the present invention can also be achieved by performing reading.

  In this case, the program itself read from the recording medium realizes the novel function of the present invention, and the recording medium storing the program constitutes the present invention.

  As a recording medium for supplying the program, for example, a flexible disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, DVD-ROM, magnetic tape, nonvolatile memory card, ROM, EEPROM, silicon A disk or the like can be used.

  Further, by executing the program read by the computer, not only the functions of the above-described embodiments are realized, but also an OS or the like operating on the computer based on an instruction of the program is a part of the actual processing or It goes without saying that the case where the functions of the above-described embodiments are realized by performing all of the above processing is also included.

  Furthermore, after the program read from the recording medium is written to the memory provided in the function expansion board inserted into the computer or the function expansion unit connected to the computer, the function expansion board is based on the instructions of the program code. It goes without saying that the case where the CPU or the like provided in the function expansion unit performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.

  Further, the present invention may be applied to a system composed of a plurality of devices or an apparatus composed of a single device. Needless to say, the present invention can be applied to a case where the present invention is achieved by supplying a program to a system or apparatus. In this case, by reading a recording medium storing a program for achieving the present invention into the system or apparatus, the system or apparatus can enjoy the effects of the present invention.

  Furthermore, by downloading and reading a program for achieving the present invention from a server, database, etc. on a network using a communication program, the system or apparatus can enjoy the effects of the present invention. In addition, all the structures which combined each embodiment mentioned above and its modification are also included in this invention.

100 Knowledge structure extraction device 130 User terminal 140 External network 201 CPU
202 ROM
203 RAM
204 System bus 205 Input controller 206 Video controller 207 Memory controller 208 Communication I / F controller 209 Input unit 210 Display 211 External memory

Claims (3)

Field classification means for classifying the document based on the document content;
Contribution calculation means for calculating a contribution for discriminating from other documents with respect to a keyword included in the document;
Unit classification means for classifying the document into units using a self-organizing map based on the contribution calculated by the contribution calculation means;
Display means for displaying the units classified by the unit classification means;
Cluster generation means for generating a cluster of each unit using hierarchical clustering based on each unit classified by the unit classification means;
Arrangement information calculating means for calculating arrangement information of the keyword with respect to the display means from a unit specified from the appearance frequency of the keyword included in the document in the unit;
Link generation display means for generating a link from the association between the cluster to which the specified unit belongs and another cluster by displaying a degree of coincidence between the specified unit and the cluster, and displaying the link on the display means;
A knowledge structure extraction device characterized by comprising: A knowledge structure extraction method for extracting and displaying related keywords from a document in a knowledge structure extraction device,
The field classification means of the knowledge structure extraction device includes a field classification step of classifying the document based on the document content;
The contribution calculation unit of the knowledge structure extraction device includes a contribution calculation step for calculating a contribution for discriminating the keyword included in the document from another document;
A unit classification step of classifying the document into units using a self-organizing map based on the contribution calculated in the contribution calculation step;
The display means of the knowledge structure extraction device displays a unit classified by the unit classification step,
The cluster generation means of the knowledge structure extraction device includes a cluster generation step of generating a cluster of each unit using hierarchical clustering based on each unit classified by the unit classification step;
An arrangement information calculating unit of the knowledge structure extracting device calculates an arrangement information of the keyword with respect to the display step from a unit specified from an appearance frequency of the keyword included in the document in the unit;
The link generation means of the knowledge structure extraction device generates and displays a link from the association between the cluster to which the specified unit belongs and another cluster by obtaining the degree of coincidence between the specified unit and the cluster. A link generation display step to perform,
The knowledge structure extraction method characterized by performing this. A program executed in a knowledge structure extraction device that extracts and displays related keywords from a document,
The knowledge structure extraction device;
Field classification means for classifying the document based on the document content;
Contribution calculation means for calculating a contribution for discriminating from other documents with respect to a keyword included in the document;
Unit classification means for classifying the document into units using a self-organizing map based on the contribution calculated by the contribution calculation means;
Display means for displaying the units classified by the unit classification means;
Cluster generation means for generating a cluster of each unit using hierarchical clustering based on each unit classified by the unit classification means;
Arrangement information calculating means for calculating arrangement information of the keyword with respect to the display means from a unit specified from the appearance frequency of the keyword included in the document in the unit;
Link generation display means for generating a link from the association between the cluster to which the specified unit belongs and another cluster by displaying a degree of coincidence between the specified unit and the cluster, and displaying the link on the display means;
A program characterized by functioning.

Images