JP2011191834A - Method, device and program for classifying document - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To output the number of classifications desired by a user by improving a reproduction rate while maintaining a relevance ratio, in a KNN (K-Nearest Neighboring) method general as a document classification method. <P>SOLUTION: A total of similarity degrees of similar documents imparted with the classification is calculated about each of the pieces of the classification imparted to K-pieces of similar documents, and the classification having a high total value is specified. Next, processing of excluding the specified classification from the classification imparted to the similar documents, excluding the similar document wherein an imparted classification number becomes zero as the result of the exclusion, specifying the K-pieces of similar documents from the left similar documents, and specifying the classification is repeated. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a document classification method, a document classification device, and a document classification program that use a document to which no classification is assigned as input and assign an appropriate classification from a predefined classification system. In particular, the present invention relates to a document classification method capable of outputting an arbitrary number while maintaining a high classification accuracy and a high recall and relevance rate, and a document classification apparatus and program for executing the document classification method.

  When searching for a desired document from a large number of documents containing text information, such as patents, papers, newspaper articles, and web pages, a classification is assigned in advance according to the document format, bibliography, and contents. A method of narrowing down the search range by specifying the classification in the search condition is effective. However, the task of assigning classifications to a large number of documents, the task of constantly updating the classification system to the optimal state according to the flow of the times, and the classification assigned to a large number of past documents when the classification is updated. The current situation is that a great deal of labor is required for repair work.

  Therefore, automatic document classification technology has attracted attention as a technology for reducing the burden on workers involved in these operations. As an automatic classification algorithm, for example, as disclosed in Japanese Patent Application Laid-Open No. 2007-323454, a document set to which classification is assigned in advance is prepared as a learning document set, and a document that has not yet been assigned classification is prepared. On the other hand, a similar learning document is searched, and the classification status assigned to the higher-ranking learning document is statistically analyzed to calculate a classification score for each classification, and according to the size of the classification score. A method called a K-Nearest Neighboring method is generally used (hereinafter, referred to as a KNN method in this specification). In the KNN method, the classification score is assigned to each classification in consideration of the classification given to many of the learning documents in the top K search results (K of KNN) and the classification given to higher learning documents. Calculate and output classification in descending order of classification score.

JP 2007-323454 A

  The KNN method is a method often used in document classification because it can use an index common to similar document retrieval, has a relatively simple algorithm, and has a relatively high classification accuracy. However, on the other hand, the number of classifications to be output becomes unstable due to the characteristics of the algorithm.

  For example, if the top 10 similar document search results are used to identify the classification and it is assumed that only one type of classification is assigned to one document, the number of classifications that can be output for one document is , At least one and at most ten. It is sufficient that the correct classification is included in this output classification. However, when the classification system is a large number of classifications and the classification granularity is small, such as patent classification, the correct classification is There is often leakage, and as a result, the recall (ratio of missing classification) may be reduced.

  In order to increase the number of classifications to be output, it is possible to increase the number of similar documents targeted by the KNN method (in the above example, 10 to 100). It has been found from the results of the evaluation experiment that the ratio of the absence is reduced.

  On the other hand, as an application of automatic document classification, there is an application that improves the work efficiency rather than manually classifying from the beginning by manually examining the automatic classification results and adding, deleting, and correcting as necessary. As a result of human inspection, if it is determined that the automatic classification result is not valid, the work efficiency is deteriorated unless the candidate classification can be presented to the user without omission in order to give an appropriate classification. Therefore, it is desirable that as many automatic classification results as possible can be output according to the user's request.

  Based on the above, in a simple and powerful document classification method called the KNN method, it becomes a problem to be able to output a certain number of classifications without reducing the recall rate and the matching rate.

  Therefore, in the present invention, the top K similar documents (K is a natural number designated in advance) searched by the similar document search means are specified for the inputted classification grant target document, and the K similar documents are specified. For each of the assigned classifications, the “number of documents” or “total value of similarities (similar document search scores)” or “similarity of the similarities in the search rankings of the K similar documents. "Sum of values divided by logarithm" is calculated, and the top N (N is a natural number designated in advance) classification with the highest calculated value is classified as a classification grant result to be assigned to the classification grant target document. A similar document stored in the assignment result table and excluding the N classifications from the classifications assigned to the similar documents searched by the similar document search means, and the number of classifications given as a result of the exclusion is zero. Excluded, excluded By repeatedly executing the process of identifying the top K similar documents in the remaining similar documents until the number of classifications stored in the classification assignment result table reaches the number specified in advance by the user, To solve.

  According to the present invention, in a simple and powerful document classification method called the KNN method, it is possible to output an arbitrary number of classifications without reducing the reproduction rate and the matching rate. In addition, since a certain number of classification grant results with high accuracy can be presented to the user, the user can scrutinize the validity of the automatic classification result more efficiently.

The figure which shows an example of the block diagram in embodiment of this invention. The figure which shows an example of the hardware constitutions in embodiment of this invention. The figure which shows an example of the process outline | summary of the classification provision part 9 in embodiment of this invention. The figure which shows an example of a structure of the similar document table 8 in embodiment of this invention. The figure which shows an example of a structure of the document-classification correspondence table 10 in the embodiment of the present invention. The figure which shows an example of a structure of the similar document table with a classification | category produced | generated and referred by the classification provision part 9 in embodiment of this invention. The figure which shows an example of the processing flow of the classification provision part 9 in embodiment of this invention. The figure which shows an example of the processing flow of the classification specific | specification part 11 in embodiment of this invention. The figure which shows an example of the processing flow of the classification edit part 12 in embodiment of this invention. The figure which shows an example of the parameter setting screen in embodiment of this invention.

  Embodiments of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited thereby.

  In the present embodiment, a document classification system will be described in which a document including text is input and an appropriate classification is automatically specified from a predefined classification system. In this embodiment, a document written in Japanese is targeted, but the present invention can also be applied to a document written in a foreign language such as English. This system searches for a document whose contents are similar to the input document from a learning document set (a set of documents with classifications assigned in advance), and is assigned to the higher-ranked document as a result of the search. Based on the KNN method of determining the classification to be given by statistical analysis of the classification assignment status. In the KNN method, a learning document set to which a classification is assigned in advance is indispensable. However, in the present embodiment, it is assumed that a learning document set already exists.

  FIG. 1 is a diagram showing an example of a block diagram in the present embodiment. Details of each data and processing unit will be described in detail in FIG.

  The document classification system receives the classification grant target document set 2 as input, identifies the classification to be given, and outputs it to the classification grant result table 13. Prior to the processing, for each document constituting the learning document set 1, the document analysis unit 3 refers to the word dictionary 4 and morphologically analyzes the text to extract weighted feature words that characterize the contents of the document, The search index generation unit 5 stores the document ID and the weighted feature word in the search index 6 in a form of associating them.

  In the automatic classification process, first, the document analysis unit 3 similarly refers to the word dictionary 4 for each document constituting the classification assignment target document set 2 and extracts a weighted feature word by performing morphological analysis on the text.

  Next, the similar document search unit 7 refers to the search index 6 to identify a learning document having a feature word set similar to the extracted weighted feature word set, and similar documents in descending order of similarity. The document is stored in the similar document table 8 together with the similarity (similar document search score).

  Next, the classification assigning unit 9 takes the similar document table 8 as an input, and refers to the document-classification correspondence table 10 describing the correspondence between the documents constituting the learning document set 1 and the classifications assigned to the documents. Thus, the classification to be assigned to the classification assignment target document 2 is specified and stored in the classification assignment result table 13.

  In this system, parts other than the classification assigning unit 9 have the same configuration as the conventional KNN method. That is, the morphological analysis in the document analysis unit 3 is already a known technique, and many tools are disclosed. The present invention is a method that does not depend on a morpheme analysis method, and any morpheme analysis tool may be used. In addition, as a method for extracting weighted feature words, there are already known methods such as TF-IDF method considering the frequency of appearance of words and the number of appearing documents, weighting considering the occurrence location of words and co-occurrence appearance tendency, etc. Technology. The present invention is a method that does not depend on the feature word extraction and weighting method, and any method may be adopted, and thus will not be described in detail here. Furthermore, there are many similar document search tools and commercial products for the feature word collation method, similarity calculation method, and search index generation method in the similar document search unit 7. In the similar document search, there are a wide variety of methods such as generating a document vector whose elements are weighted feature words of each document and determining that the higher the cosine value of the angle between the document vectors of the two documents, the more similar to each other. Are known. The present invention only needs to be able to search for a similar document with a similarity (similar document search score), and does not depend on the content of the similar document search method, and therefore will not be described in detail here.

  In this invention, the classification | category provision part 9 is a part used as the point of invention. The classification assigning unit 9 includes two processing units, that is, a classification specifying unit 11 and a classification editing unit 12. By repeatedly executing these processing units alternately, the classification is sequentially specified.

  The classification specifying unit 11 refers to the similar document table with classification obtained from the similar document table 8 and the document-classification correspondence table 10, specifies the top N classifications to be assigned, and stores them in the classification assignment result table 13. When the number of classifications stored in the classification assignment result table 13 reaches a value designated in advance by the user, the processing of the classification assignment unit 9 is terminated. Regarding the top N classification methods, the “number of cases” and “sum of similarities” of similar documents to which a certain classification is assigned are targeted for the top K similar documents stored in the similar document table with classification. , Any one of “the sum of values obtained by dividing the similarity by the logarithm of“ search order + 1 ”” is calculated, and the top N (N is a natural number designated in advance) having the highest calculated value is specified.

  The classification editing unit 12 excludes the classification stored in the classification assignment result table 13 at this time from the classifications assigned to the similar documents stored in the classified similar document table. When the number of classifications assigned to the similar document becomes 0 as a result of the exclusion, the document is excluded from the similar documents, and the search order is sequentially increased up. By this processing, the top K similar documents are dynamically changed.

  FIG. 2 is a diagram illustrating an example of a hardware configuration of the present embodiment. This system is large and includes a processing device 50 for executing calculation processing, a keyboard 51 and a mouse 52 for a user to input operation contents or data, an output monitor 53 for outputting a calculation processing result to the user, and a processing device 50. It is comprised from the memory | storage device 60 which stores the program and data regarding the processing in. When the input / output data is exchanged with another computer, the input / output data is transmitted / received via the network 54.

  The storage device 60 further includes a working area 61 for temporarily storing processing data in the processing device 50, a learning document set storage area 62, a classification grant target document set storage area 63, a document analysis unit storage area 64, and a word dictionary storage. Area 65, search index generation unit storage area 66, search index storage area 67, similar document search unit storage area 68, similar document table storage area 69, classification assigning unit storage area 70, classification specifying unit storage area 71, classification editing unit storage An area 72, a document-classification correspondence table storage area 73, and a classification assignment result table storage area 74 are configured. The processing device 50 performs processing by repeatedly loading necessary programs and data from the storage device 60 and storing the execution results in the storage device 60.

  FIG. 3 is a diagram showing an outline of processing of the classification assigning unit 9. FIG. 3 (1) shows data in the initial state of the similar document table with classification generated and referred to by the classification assigning unit 9. The similar document table 8 and the document-classification correspondence table 10 are joined (joined with the similar document ID as a key). ). This data includes similar document search result ranking 301, similar document ID 302 searched as a document similar to the input document, similarity 303 obtained by quantifying the degree of similarity with the input document, and classification 304 assigned to the similar document. It is configured. In FIG. 3, one classification is assigned to one document. However, even if a plurality of classifications are assigned to one document, the same processing can be performed.

  Based on the KNN method, N classifications (one in FIG. 3) are identified based on the top K similar documents (10 in FIG. 3) of the initial state data. As the method for specifying the classification, the following three types are conceivable, but other methods may be used.

  (1) The categories are arranged in the descending order of the number of similar documents in the top K to which the classification is given, and the top N items are specified as the classification to be given.

  (2) The sum of the similarities of the similar documents in the top K cases to which the classification is given is calculated, the classifications are arranged in descending order of the sum, and the top N items are specified as the classifications to be given.

(3) The similarity of similar documents in the top K documents to which classification is assigned is “search order + 1”
The sum of the values divided by the logarithm of is calculated, and the categories are arranged in descending order of the sum, and the top N items are specified as the categories to be assigned.

  In this embodiment, K = 10 and N = 1, but these parameters can be easily specified by the user. Further, although the number of similar documents output as the similar document search result is 10,000, it is easily possible for the user to specify this.

  In FIG. 3 (1), if the classification C01 is specified as a classification to be given by the above-described method, all the classifications to which C01 is assigned among the classifications 304 given to similar documents are next. exclude. As a result of the exclusion, if any similar document without any classification is created, the similar document itself is excluded. Blank spaces created by excluding similar documents are filled in by increasing the order. Data obtained by excluding the classification C01 is FIG. 3 (2). The similar document 0004 ranked first in the initial state is excluded, and the second similar document 0015 is upgraded to the first. The total number of similar documents has decreased from 10,000 to 9000. This indicates that the number of similar documents excluded by excluding classification C01 was 1000.

  For the top K similar documents in this state, N classifications are determined again by the above-described KNN method. If the classification C02 is determined as a classification to be given, the classification C02 is excluded and the rank is increased.

  The above operation is repeated, and the process is terminated when the number of specified classifications reaches the number designated in advance by the user. The classifications to be given are ranked in the order determined by the above method (classification C01, classification C02,...).

  In this way, by dynamically changing the range of similar documents for which classification is specified by the KNN method and sequentially specifying the classification to be given, the reproduction rate is improved without reducing the relevance rate. The user can output a desired number of classifications.

  FIG. 4 is a diagram showing a data configuration of the similar document table 8 for the classification grant target document set 2. The input document ID 401 uniquely identifies the input document, the similar document search order 402 for the document, the similar document ID 404 that uniquely identifies the similar document, and the similarity 403 of the similar document with respect to the input document. In the present embodiment, only documents constituting the learning document set 1 to which classification is assigned in advance are output as similar document search, but similar from a document set including documents to which no classification is assigned. A similar document table 8 may be generated by searching for documents and filtering only learning documents to which classifications have been assigned in advance from the search results (in this case, a statistical value such as the number of appearing documents used in similar document search may be determined). Will result in different search results).

  FIG. 5 is a diagram showing a data configuration of the document-classification correspondence table 10. A document ID 411 and a classification 412 assigned to the document are stored as a pair. A plurality of classifications may be assigned to one document as document ID = 1345, 00002.

  FIG. 6 is a diagram showing the structure of a similar document table with classification generated and referred to by the classification assigning unit 9, and corresponds to the data described in FIG. This data is stored in the working area 61. The classification 425 of the similar document ID 424 output as the similar document search result is described in association with it. When a plurality of classifications are assigned to one similar document, such as the similar documents of the first and third search rankings 422, a plurality of classifications such that one classification is included in the classification 425 of the similar document ID. Describe the records separately.

  FIG. 7 is a diagram illustrating a processing flow of the classification assigning unit 9. First, the similar document table 8 and the document classification correspondence table 10 are joined using the similar document ID as a key, thereby generating the similar document table with classification described in FIG. 6 and storing it in the working area 61 (step) 1001). Next, the processing of the classification specifying unit 11 is executed (step 1002). Next, it is determined whether or not the number of classifications stored in the classification assignment result table 13 has reached the number designated in advance by the user (step 1003). If not reached, the process of the category editing unit 12 is executed (step 1004), and the process returns to step 1002.

  FIG. 8 is a diagram showing a processing flow of the classification specifying unit 11 constituting the classification giving unit 9. First, the top K similar document IDs 424 are specified in the classified similar document table generated in step 1001 of the classification assigning unit 9 and stored in the working area 61 (step 2001). Next, a classification list composed of “similar document ID classification 425” assigned to the identified top K similar document IDs 424 is generated (step 2002). Next, for each of the classifications constituting the classification list, the similar document IDs 424 in the top K items to which the classification is assigned are specified, and the sum of the similarities 423 of the specified similar document IDs 424 is calculated (step 2003). In this step, instead of calculating “the sum of the similarities 423 of the specified similar document IDs 424”, the “number of specified similar document IDs 424” or the “similarity of the specified similar document ID 424” is searched. The sum of the values divided by the logarithm of “rank +1” ”may be calculated. Next, the “similarity of similarity” for each classification calculated in step 2003 is sorted in descending order (step 2004). Next, the top N classifications with a high sum of similarities are stored in the classification assignment result table 13 (step 2005).

  FIG. 9 is a diagram illustrating a processing flow of the classification editing unit 12 constituting the classification assigning unit 9. First, the value of the counter C that counts the number of rows in the classified similar document table is initialized to 1 (step 3001). Next, it is determined whether or not data exists in the C-th row of the similar document table with classification (step 3002). If it exists, any of the classifications stored in the classification assignment result table 13 at this time is determined. Then, it is determined whether or not it matches the “similar document ID classification 425” of the C line of the similar document table with classification (step 3003), and if it matches, the record of the C line is deleted and the C + 1 and subsequent lines are deleted. Line up (step 3004). According to this step, a similar document in which the number of classifications assigned to one similar document becomes zero is automatically excluded. If they do not match at step 3003, 1 is added to the counter C (step 3005), and the process returns to step 3002. If it is determined in step 3002 that data does not exist, the search order 422 of the classified similar document table is reassigned in order from 1 (step 3006). That is, as a result of deleting records in step 3004, the items whose search order is skipped are shaped so as to be in order from 1. Here, the same search order 422 is assigned to the same line with the similar document ID 424.

  FIG. 10 is a diagram showing an example of a parameter setting screen for inputting various parameter settings used in the present system. Parameter setting on this screen may be performed by an end user, or may be performed by a person other than the end user, such as a system administrator. This setting is set before executing the classification process, but it may be set as appropriate by looking at the classification result.

  In this embodiment, the parameters set on this screen are as follows.

(1) Total number of similar documents output by the similar document search unit 7 This is a value corresponding to 10000 in FIG.

(2) Number of similar documents used when specifying a classification A value corresponding to K in FIG. 8 and a value corresponding to 10 in FIG.

(3) Number of classifications given in one process by the classification specifying unit 11 A value corresponding to N in FIG. 8 and a value corresponding to one in FIG.

(4) Number of output classifications This is the number of classifications output as a classification result.

(5) Score calculation method This is a score calculation method for calculating which classification should be given from the similarity of the top K similar documents. In this embodiment, one of the following three types is used. select.

(A) Number of similar documents in top K to which classification is assigned (b) Sum of similarities of similar documents in top K to which classification is assigned (c) Top K to which classification is assigned The similarity of similar documents in the case is set to “search order +1”
The sum of the values divided by the logarithm of is stored in the working area 61 with the parameter values set on this screen, and the classification process is executed by referring appropriately from each processing unit.

50 ... processing device, 60 ... storage device, 51 ... keyboard, 52 ... mouse, 53 ... output monitor, 54 ... computer network.

Claims (3)

A learning document set, a document-classification correspondence table in which each document constituting the learning document set and a classification previously assigned to the document are described in association with each other, and a classification grant target document that has not yet been assigned a classification Based on the classification assigned to the retrieved similar documents with reference to the document-classification correspondence table, and the similar document retrieval means for retrieving documents similar to the classification assignment target document from the learning document set. A document classification method in a document classification apparatus having a classification providing means for determining a classification to be assigned to the classification grant target document,
The top K similar documents (K is a natural number designated in advance) searched by the similar document search means are specified, and for each of the classifications assigned to the K similar documents, the K similar documents are identified. Among them, either “Number of documents”, “Sum of similarity (similar document search score)” or “Sum of values obtained by dividing similarity by logarithm of“ search rank + 1 ”” To calculate
The top N classifications (N is a natural number designated in advance) having the highest calculated value are stored in the classification assignment result table as classification assignment results to be assigned to the classification assignment target document,
The N classifications are excluded from the classifications assigned to the similar documents searched by the similar document search means, and the similar documents in which the number of classifications given as a result of the exclusion are 0 are excluded and are not excluded. The process of identifying the top K similar documents in the remaining similar documents
Repeatedly until the number of classifications stored in the classification assignment result table reaches a number specified in advance by the user,
Document classification method characterized by the above. A learning document set, a document-classification correspondence table in which each document constituting the learning document set and a classification previously assigned to the document are described in association with each other, and a classification grant target document that has not yet been assigned a classification Based on the classification assigned to the retrieved similar documents with reference to the document-classification correspondence table, and the similar document retrieval means for retrieving documents similar to the classification assignment target document from the learning document set. In the document classification apparatus having a classification providing means for determining a classification to be assigned to the classification grant target document,
For each of the classifications assigned to the top K similar documents (K is a natural number designated in advance) searched by the similar document search means, among the K similar documents, the documents assigned the classification Calculate either “Number of cases” or “Sum of similarity (similar document search score)” or “Sum of similarity divided by logarithm of“ search rank + 1 ””
The top N classifications (N is a natural number designated in advance) having the highest calculated value are stored in the classification assignment result table as classification assignment results to be assigned to the classification assignment target document,
The N classifications are excluded from the classifications assigned to the similar documents searched by the similar document search means, and the similar documents in which the number of classifications given as a result of the exclusion are 0 are excluded and are not excluded. The process of identifying the top K similar documents in the remaining similar documents
Means for alternately and repeatedly executing until the number of classifications stored in the classification assignment result table reaches a number specified in advance by the user;
A document classification device characterized by that. A learning document set, a document-classification correspondence table in which each document constituting the learning document set and a classification previously assigned to the document are described in association with each other, and a classification grant target document that has not yet been assigned a classification Based on the classification assigned to the retrieved similar documents with reference to the document-classification correspondence table, and the similar document retrieval means for retrieving documents similar to the classification assignment target document from the learning document set. A program having a computer having a classifying unit for determining a class to be given to the document to be classified,
The top K similar documents (K is a natural number designated in advance) searched by the similar document search means are specified, and for each of the classifications assigned to the K similar documents, the K similar documents are identified. Among them, either “Number of documents”, “Sum of similarity (similar document search score)” or “Sum of values obtained by dividing similarity by logarithm of“ search rank + 1 ”” To calculate
The top N classifications (N is a natural number designated in advance) having the highest calculated value are stored in the classification assignment result table as classification assignment results to be assigned to the classification assignment target document,
The N classifications are excluded from the classifications assigned to the similar documents searched by the similar document search means, and the similar documents in which the number of classifications given as a result of the exclusion are 0 are excluded and are not excluded. The process of identifying the top K similar documents in the remaining similar documents
Causing the computer to repeatedly execute until the number of classifications stored in the classification assignment result table reaches a number specified in advance by a user,
A program characterized by that.

Images