JP4534019B2 - Name and keyword grouping method, program, recording medium and apparatus thereof - Google Patents

Description

  The present invention automatically associates a person's name with a keyword representing a research field, which is useful in finding a person related to a certain research field, technical field, organization, interest, etc. as a query. By grouping them into groups, groups of researchers can be categorized autonomously from information on the Web, and it is possible to make it easier for users to grasp appropriate research fields as queries from the obtained groups. It relates to methods.

  The inventors of the present invention have already proposed a method for automatically extracting human relationships from information on the Web (see Non-Patent Document 1). In this method, a web page including the names of two persons to be searched is searched, the strength of the relationship between the two names is obtained from the number of hits, and the human relationship between the two is extracted based on that, and illustrated as a network Can do. This network expresses human relationships as an undirected graph in which nodes represent people, and the edges connecting the nodes represent the relationship between the two, and the relationship between the two by the labels attached to the edges For example, in the case of researchers, you can understand co-authorship relationships, relationships that are members of the same laboratory, relationships that have been announced at the same research meeting, relationships that are researched in the same project, etc. It can also be seen that the shorter the edge, the stronger the relationship.

Such a human relationship network is useful for overlooking human relationships within a certain field, area, or organization. For example, if you want to get acquainted with Mr. A at a certain academic society, check the shortest path between you and Mr. A in the network of human relations extracted for that school member, or find a common acquaintance between yourself and Mr. A. Thus, it is possible to clearly understand the human relationship with other members and the like surrounding both names, and thus it is possible to easily find an appropriate person who is likely to introduce Mr. A.
Yutaka Matsuo, Hironori Tomobe, Koichi Hashida and Mitsuru Ishizuka, "Mining Social Network of Conference Participants from the Web," 12th International WWW Conference, poster, 2003 Henry Kautz, Bart Selman, and Mehul Shah, "Referral Web: combining social networks and collabolative filtering," Communications of the ACM, vol. 40, no. 3, 1997 H. Kautz, B. Selman, and A. Milewski, "Agent amplified communication," Proceedings of the National Conference on ArtificialIntelligence, pp.3-9, 1999 H. Kautz, B. Selman, and M. Shah, "The Hidden Web," AIMagazine, 18 (2), pp.27-36, 1997 G. Salton and MJ McGill, "Introduction to ModernRetrieval," McGraw-Hill Book Company, 1983 IS Dhillon, "Co-clustering documents and words using Bipartite Spectral Graph Partitioning," Proceedings of the Seventh ACMSIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), 2001

  However, for example, when it is desired to find an expert in a specific field or the like, the user needs to know the name of the expert in advance in the human relationship network. This is because the node is the name of a person and can only be searched for by name.

  In this case, it is necessary to be able to search for a person related to the field from the field, and in order to achieve this, by labeling the field related to the person for each name, It is thought that people can be searched from the label. For example, in the case of a researcher, by giving a research field to each researcher, a researcher in that field can be searched from the research field.

  However, the field of research varies depending on the context in which the person is involved. For example, even if a researcher is recognized as a “natural language processing” researcher among researchers working on “artificial intelligence”, “syntactic analysis” is more detailed among researchers working on “natural language processing”. It is sometimes recognized as a researcher, and the research field is determined by the context.

  Therefore, after looking at the entire target researcher, it is important to determine the level of granularity of the research field to make an efficient database or search without omissions. Conceivable. In the above example, the more efficient search is possible, when the research field is determined by the granularity of “natural language processing” or when the research field is determined by the granularity of “syntactic analysis” in more detail That's what it means.

  That is, it is necessary to determine the granularity to be used for grouping and then perform grouping.

  Therefore, in view of the circumstances as described above, the present invention can appropriately and automatically group a keyword group in a research field or the like and a name group such as a person related thereto as described above. The problem is to propose a method for grouping names and keywords.

  The present invention solves the above-mentioned problem. First, the processing unit accepts an input from the name and keyword input unit, and the processing unit receives the co-occurrence matrix of the input name and keyword. And a grouping method of names and keywords, wherein the processing unit has a step of clustering names and keywords in the created co-occurrence matrix.

  Second, the name and keyword grouping method is provided, wherein the processing unit creates the co-occurrence matrix using the number of public data including both the name and the keyword.

  Third, the name and keyword grouping method is provided, wherein the processing unit weights the number of public data.

  Fourth, the processing unit accepts an input from an input unit of a label representing each group obtained by the clustering, and the processing unit stores the input label in association with each group. The name and keyword grouping method is further provided, further comprising the step of storing in a means.

  Fifth, a name and keyword grouping program for causing a computer to execute the name and keyword grouping method is provided.

  Sixth, a computer-readable recording medium on which the name and keyword grouping program is recorded is provided.

  Seventh, means for inputting a name and a keyword, means for creating a co-occurrence matrix of the input name and keyword, and means for clustering names and / or keywords in the created co-occurrence matrix Provided is a name and keyword grouping device characterized by comprising the above.

  Eighth, the name and keyword grouping apparatus is characterized in that the means for creating a co-occurrence matrix creates the co-occurrence matrix using the number of public data including both the name and the keyword. To do.

  Ninth, the name and keyword grouping device is provided, wherein the number of public data is a weighted number of public data.

  Tenth, the name and keyword further comprising means for inputting a label representing each group obtained by the clustering, and means for storing the input label in association with each group. A grouping device is provided.

  According to the name and keyword grouping methods of the first to fourth aspects of the present invention, the name and keyword co-occurrence matrix is created and clustered to realize the name and keyword grouping. By referring to this keyword group, a label can be assigned to a group of people corresponding to the keyword group.

  That is, groups of people can be categorized autonomously from information published in various databases represented by the Web.

  As a result, for example, when researchers and research fields are grouped, it is possible to search for researchers belonging to the group at a time using the group name, ie, research category, assigned to each research field group as a query. Since they are presumed to be experts in the field, as described above, the related experts can be comprehensively searched from the coarse-grained keywords representing the field.

  For clustering, a method called co-clustering (see Non-Patent Document 6) that can simultaneously obtain a name cluster (= group; the same applies hereinafter) and a keyword cluster, and obtain a name cluster and a keyword cluster separately. A K-means method, a minimum distance method, a maximum distance method, or the like can be used. In any case, a cluster of people and a cluster of keywords corresponding to the cluster can be obtained. Further, there is no particular limitation as long as it is a clustering technique that enables this.

  According to the grouping program and the recording medium of the fifth and sixth inventions of the present invention, a computer program capable of obtaining the same effects as those of the first to fourth grouping methods, and a flexible disk or CD on which the computer program is recorded , A recording medium such as a DVD is realized, and according to the grouping devices of the seventh to ninth inventions of the present invention, a device that can achieve the same effects as the first to fourth grouping methods is realized.

  Note that several studies have been conducted to discover experts using information on the Web. For example, an information search system called Referral web using a social network is known (see Non-Patent Documents 2 to 4). In this system, given a user's name, the social network around the user is displayed. Each user has an agent and the agent finds experts on the social network who have topic words that match the user's interests.

  While this Referral web focuses on each user and extracts each topic word, the present invention first grasps the entire target person and based on the co-occurrence with keywords in it. This is an important feature.

  That is, the co-occurrence matrix of the name and the keyword is calculated, and it is calculated based on the number of public data such as the number of Web pages including both the name and the keyword, that is, the strength of the relationship between the name and the keyword is Definition based on co-occurrence in various databases, etc., and weighting the co-occurrence matrix, that is, expressing the name as a keyword weighting vector, is a feature of the present invention that has never existed before. This makes it possible to appropriately and automatically group names and keywords in association with each other.

  In addition, as a name targeted by the present invention, not only a person's name but also various kinds of names such as a company name, an organization name, an occupation name, a product name, an animal and plant name, a book name, a song name, a country name, and a place name can be considered. In the same way as person names, grouping of various names and related keywords can be realized. Moreover, not only proper nouns but also various common nouns can be regarded as names and grouped with keywords, and any word or word can be targeted.

  Hereinafter, specific processing of grouping will be described in more detail with reference to FIGS. 1 and 2 as appropriate. FIG. 1 is a flowchart of the grouping process, and FIG. 2 is a system configuration diagram of a grouping apparatus that executes the grouping process.

  In the system configuration of FIG. 2, a display unit (1), an input unit (2), a processing unit (CPU) (3), a storage unit (memory) (4), a communication control unit (5), a matrix database (6) and A bus (7) is provided. The storage unit (4) stores a grouping program and various data, and the processing unit (3) connected to the storage unit (4) and the bus (7) receives an instruction of the grouping program. To execute the grouping process. In addition, the processing unit (3) includes both a display unit (1) such as a display for displaying an input screen and various data, and an input unit (2) such as a keyboard and a mouse for inputting names, keywords, and the like. ). The matrix database (6) stores the co-occurrence matrix calculated by the processing unit (3).

<Step S1>
First, the processing unit (3) receives input of names and keywords to be grouped from the input unit (2).

  More specifically, for example, in order to group researchers belonging to a certain academic society with their research contents, select the target researcher from the academic membership list published on the web or the national convention program. , Enter each name. In addition, keywords that are associated with research content are selected from published papers and entered. The processing unit (3) receives the input from the input unit (2).

  For keywords, a method of automatically extracting from a paper using a known keyword extraction technique can be considered. In the case of this automatic extraction, even if the processing unit (3) executes the extraction process according to its own keyword extraction program, the processing unit (3) puts it on the Web page to a separate keyword extraction engine or the like accessible via the network (8). An extraction process may be executed on a published paper and the result may be received.

<Step S2>
Next, a co-occurrence matrix of the input name and keyword is created by the processing unit (3).

Conventionally, in information retrieval, the entire document set is often represented as a document-word matrix. This is an approach well known as a vector space model (see Non-Patent Document 5). In the vector space model, content words are extracted from all document sets as features, and each document is expressed as a weight vector representing the depth of the relationship with the features. In general, the weight w _ij of the word j in the document i is calculated using a tfidf (term frequency inverse document frequency) method as shown in the following equation.

Here, tf _ij is the appearance frequency (term frequency) of word j in document i, df _j is the number of documents in which word j appears (document frequency), and N is the total number of documents. Qualitatively, because of the effect of tf, the more frequently the word appears in the document, the higher the weight, and the effect of idf (inverse document frequency) increases the weight in the document as the word appears locally in a specific document. Will grow.

  Similarly, let us consider representing a name as a keyword weight vector. If the subject is a researcher, the fact that a researcher's name often co-occurs with a research keyword is considered to indicate that there is some relationship between the researcher and the research keyword. Therefore, the strength of the relationship between the researcher and the research keyword is defined based on the co-occurrence of the researcher name and the research keyword on the Web. Appearing in the same web page is called co-occurrence. If AND search of the researcher name and research keyword is performed, the number of hit web pages is the number of web pages where this researcher name and research keyword co-occur. It is. By executing this for each researcher name and research keyword, a co-occurrence matrix can be created in which the name is horizontal (row) and the keyword is vertical (column) (or vice versa).

  Here, if the number of co-occurrence Web pages is used as a weight as it is, a phenomenon that a relatively general research keyword is heavily weighted may occur.

More specifically,
1. The distribution of researchers with few co-occurrence cases cannot be measured accurately.
2. For researchers with a large number of co-occurrence cases, the number of co-occurrence cases with words that are not much related is also relatively large.
3. When a general word is included, the weight of the column becomes relatively large.

  Therefore, it is preferable to calculate the statistical bias of co-occurrence rather than the simple number of co-occurrence hits.

For example, the weight w _ij of the research keyword j for the researcher i is defined by the analogy of Equation 1 as follows.

Where co _ij is the number of Web pages where researcher name i and research keyword j co-occur, pf _j is the number of researchers co-occurring with research keyword j (referred to as person frequency), and N is all researchers Is a number.

  If the weight is calculated according to this equation 2, the researcher is represented by the weight vector of the research keyword, and a researcher name-research keyword co-occurrence matrix is obtained.

In addition, as a statistic indicating the strength of bias, there is a χ ² value as one of the general ones, and the strength of co-occurrence bias can be expressed by this χ ² value to weight research keywords. .

The χ ² value is formulated as follows:

However, in the above equation, even when O _ij -E _ij becomes negative, the χ ² value takes a positive value. However, since the purpose here is to weight the researcher name with a keyword related to the person's research field, the χ ² value when the observed value is smaller than the expected value is set to zero. That is, the following equation is obtained.

  Note that these weighting methods are one of methods for measuring statistically significant bias. Various other weighting methods, for example, a method in which each element of a matrix is normalized as a probability to obtain cosine and mutual information, and between matrices It is also possible to adopt a method that measures the degree of similarity with the Kullback-Leibler statistic.

  In the above processing, regarding the search, the processing unit (3) executes the search by transmitting the name and keyword to the search engine on the separate network (8) even if it executes its own search program. The search result may be received from the search engine via the network (8).

  The number of web page hits and the co-occurrence matrix are stored in storage means such as the storage unit (4) or a separate matrix database (6) in association with each name and keyword.

By the way, since there are many people with the same name and the same name on the Web, there is a possibility that a Web page other than the target person will be searched. Therefore, the search accuracy can be improved by adding a word for identifying the person together with the person name to the search query. For example, when an organization name such as a company name, a research institution name, or a university name is considered as a word for identifying a person, the name N and the “and” of the organization name A are used as a search query. Also, if you have multiple affiliations, if your affiliation changes, or if your organization has multiple names or abbreviations,
The name N and (affiliation name A or affiliation name B or affiliation abbreviation C) is used as a search query. Of course, the person discriminating word is not limited to a word representing affiliation information such as an organization name, but may be any word that can discriminate a target person from the same name and the same name.

  For example, if the target person has multiple affiliations, or if the affiliation changes, it is possible to determine what research topics were in the past affiliation, You can find out what research topics the affiliation is divided into.

As for names, there are cases where both formal names and their abbreviations exist (especially in the case of organization names such as companies, there are many cases). For example, when searching with one name X1 such as a formal name, it is a hit. If the word Y1 included in the document to be searched and the word Y2 included in the document hit when searched by another name X2 such as an abbreviation are closely related to each other, the X1 and X2 are the names of the same organization By performing the determination process, it is possible to further improve the search accuracy.
In the above description, a Web page published on the Web is a search target, and the number of search hits is referred to as “the number of Web pages”. It goes without saying that data such as various document data can be considered as a search target, and can be referred to as “number of public data” as a broader concept.

  For public data, not only data that is widely open to the public, but also a group of data stored in a specific database (for example, a database that is not open to the public and can be accessed only within an organization) And can.

<Step S3>
Subsequently, the co-occurrence matrix is clustered by the processing unit (3).

  Here, a case where names and keywords are clustered simultaneously will be described. As a clustering method in this case, co-clustering (see Non-Patent Document 6) can be used. Co-clustering is a clustering algorithm to obtain the document and the word of the cluster at the same time, if determines the number k of clusters to find, even if you do not know whether in advance what kind of cluster is present, and automatically the document cluster, it Corresponding word clusters (and vice versa) can be obtained.

Consider that a document-word matrix is represented as a graph in which a document or a word is a vertex, and if the weight of the word in the document is not 0, an edge is drawn between them. Then, since there is no edge between documents and no edge between words, the graph becomes a bipartite graph. The basic idea of Co-clustering is to find a vertex division method (V ₁ , V ₂ ,..., V _k ) that minimizes the sum of the weights E of edges across clusters (called cut). is there. This is formulated as follows.

  In the above discussion, co-clustering can be applied to the above co-occurrence matrix by replacing the document with the name and the word with the keyword. As a result, for example, it is possible to simultaneously obtain a cluster of researchers and a cluster of research keywords corresponding to them without knowing in advance what research fields are available. However, in co-clustering, one vertex does not belong to multiple clusters, so one researcher is assigned to one research field.

  As a result, as illustrated in FIG. 3, a cluster of names (that is, name groups) and a corresponding cluster of keywords (that is, keyword groups) are obtained.

<Step S4>
After that, if a group name label suitable for expressing each content is attached to each keyword group, and it is associated with each other together with the name group and the keyword group, the label is attached to the label as a query. It becomes possible to comprehensively search for people belonging to the specified keyword group.

  In the above example, by looking at the keyword group that results from co-clustering, you can determine whether you should label "natural language processing" or more specifically "syntactic analysis". become.

  Note that the label input from the input unit (3) is stored in a storage unit such as the storage unit (4) or a separate matrix database (6) in association with each calculated cluster. Good. The processing unit (3) can automatically extract an appropriate label from, for example, the degree of association (for example, co-occurrence relationship) between an existing research field label and a research group.

<Single clustering of names and keywords in co-occurrence matrix>
By the way, in the above steps S3 to S4, as clustering for the co-occurrence matrix, co-clustering capable of simultaneously obtaining a name cluster (herein referred to as a human cluster) and a keyword cluster is described. However, if a cluster of people and a corresponding keyword cluster can be obtained, the same effect can be obtained even if other clustering methods are applied.

  For example, it is possible to cluster only people using a general clustering method such as the K-means method, the minimum distance method, or the maximum distance method. In this case, in order to find a cluster of keywords corresponding to a human cluster, for example, a centroid vector (expressed as a keyword weight vector) of the human cluster is calculated, and keywords having a large weight in that vector are clustered. To do. According to this, as in the case of co-clustering, a cluster of people and a cluster of keywords corresponding to the cluster are obtained.

  Similarly, the method of clustering only the keywords first and clustering the people with large weights in the centroid vectors (represented as the human weight vectors) of each cluster is also the keyword cluster and the corresponding human cluster. Can be obtained.

  Here, the actual grouping was performed for the participants of a research meeting held by a certain academic organization, and the results will be explained.

  First of all, the names of 200 participants from the beginning in the “Aiueo order” among the participants were targeted.

  For keywords, nouns are extracted from all published paper titles by morphological analysis, their tfidf values are calculated, arranged in descending order, and the meanings of words that are too general and unique system names are reversed. Except for too specific words, the remaining 170 words were targeted.

  The co-occurrence of these names and keywords on the Web was examined using a known search engine, the weights were calculated according to the equation 2, and a 200 × 170 co-occurrence matrix was created. The co-occurrence matrix was co-clustered and divided into 10 clusters.

  Table 2 shows the results, and shows the top 10 keywords and the number of people (that is, the number of names) in each group # 1-10. The top 10 keywords are the 10 research keywords in descending order of the sum of the weights in each cluster. In a cluster with less than 10 research keywords, all keywords are displayed.

  From this Table 2, for example, the research field of cluster 3 can be labeled “ubiquitous”, the research field of cluster 4 can be labeled “cognitive science”, and other fields can also be labeled appropriately. be able to. In addition, instead of assigning each researcher to a research field, co-clustering features that researchers with similar keyword co-occurrence patterns are grouped into a single cluster, and as a result, each cluster represents a research field. Can be said to appear well.

  For example, when cluster 3 is labeled as “ubiquitous”, each research field of the researcher cluster (21 persons) for this keyword cluster is “ubiquitous”. Similarly, all clusters are assigned labels that appropriately represent their contents. After that, if this label, that is, a research category is searched as a query, a researcher name belonging to the corresponding cluster can be acquired.

  The above examples are intended for researchers and research fields. Of course, the present invention is not limited to this, and can be applied to any person, field, region, organization, etc. Needless to say, for example, for a writer and related keywords, a keyword group such as “trick, astonishing ending, full reasoning” is labeled as a “mystery writer” for a group of people corresponding to it, or “Edo From the keyword group such as “Bakumatsu, Shinsengumi”, a group of people corresponding to the keyword group can be labeled “fictional novelist”.

  Further, in the present invention, as described above, various types of names can be targeted in addition to a person's name. For example, in the case of a company, a company name as a name, a word expressing a business type or development technology as a keyword, In the case of a product, a word representing a product name as a name and a product type or function as a keyword can be grouped.

  As described above in detail, according to the present invention, from the information published in various databases represented by the Web, by creating a co-occurrence matrix and its clustering, all kinds of things such as people, companies, products, animals and plants, etc. autonomously. A computer program that can categorize a group, a recording medium thereof, a computer device, or the like is realized.

The processing flowchart for demonstrating grouping by this invention. 1 is a system configuration diagram of a grouping device (also called a cluster creation device) according to an embodiment of the present invention. The figure for demonstrating the clustering of the name and keyword by co-clustering.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Display part 2 Input part 3 Processing part 4 Memory | storage part 5 Communication control part 6 Matrix database 7 Bus 8 Network

Claims (9)

Processing unit, with only input or we receive an input of a plurality of name i and keywords j to be grouped and stored in the storage unit step,
Creating said processing unit, and each name i and keyword j stored, the co-occurrence matrix each name i and keyword j each such name i and keyword j obtained by the AND search is made of the number of public data co-occurring The storage unit stores the name i and the keyword j in the stored co-occurrence matrix and the vertex (name i and keyword j that minimizes the sum cut of the weights E of the edges across the clusters). ) Division method (V ₁ , V ₂ ,... V _k )

A method for grouping names and keywords, comprising the step of clustering by calculating . The processing unit uses each stored name i and keyword j and the number of public data,

By calculating, the name and a keyword grouping method according to claim 1, characterized in that the weighting against the public number of data.   The processing unit uses each stored name i and keyword j and the number of public data,

The name and keyword grouping method according to claim 1, wherein weighting is performed on the number of public data by calculating. A step in which the processing unit receives an input from an input unit of a label representing each group obtained by the clustering; and a step in which the processing unit stores the input label in association with each group in a storage unit. The name and keyword grouping method according to claim 1, further comprising:   5. A name and keyword grouping program for causing a computer to execute the name and keyword grouping method according to claim 1. Means for inputting a plurality of names i and keywords j to be grouped ;
Means for storing each input name i and keyword j;
Means for creating each name i and keyword j stored, the co-occurrence matrix each name i and keyword j each such name i and keyword j obtained by the AND search is made of the number of public data that co-occur,
Means for storing the number of public data and a co-occurrence matrix;
The name i and keyword j in the stored co-occurrence matrix and the vertex (name i and keyword j) division method (V ₁ , V ₂ ,... V ) that minimizes the sum cut of the edge weights E across the clusters. _k )

A device for grouping names and keywords, characterized by comprising means for clustering by calculating . Using each stored name i and keyword j and the number of public data,

7. The name and keyword grouping apparatus according to claim 6 , further comprising means for weighting the number of public data by calculating .   Using each stored name i and keyword j and the number of public data,

7. The name and keyword grouping apparatus according to claim 6, further comprising means for weighting the number of public data by calculating. Means for inputting a label representing each group obtained by the clustering, and an input label to any one of 6 to claim and further comprising a means for storing in association with each group 8 Device for grouping names and keywords.

Images