JP4926198B2 - Method and system for generating a document classifier - Google Patents

Description

  The present invention relates to information retrieval (IR) and text data mining, in particular by combining the data distribution underlying the unclassified document set and the semantic information contained in the term category name, The present invention relates to a method and system for generating a document classifier for automatic document classification that provides highly accurate document classification.

  In recent years, the rapid growth of available electronic documents has allowed ordinary people to understand and use such large amounts of information. It is an interesting task that helps users organize large amounts of information and find interesting parts in an effective and efficient way.

  Information retrieval (IR) is the science of retrieving information in a document collection. Information Retrieval (IR) further searches for a piece of information contained in a document, searches the document itself, searches for metadata describing the document, a stand-alone relational database, text It can be divided into searching for a sound, image or data in a database such as a hypertext database networked on the Internet or an intranet. Text data mining is generally a technique related to a processing procedure for constructing high-quality information from plain text, and is divided into text classification, text clustering, concept / entity extraction, document summarization, and the like. Information retrieval and text data mining are considered to have high commercial value because the most useful information is generally stored as text or documents. Document classification is the task of classifying (labeling) natural language text having subject categories from a predefined set, eg IR and text such as word sense disambiguation, document organization, text filtering and web page search. It can be applied to many usages of data mining.

  The increasing availability of electronic information determines the importance of information retrieval and text data mining. Automatic document classification is one of the basic technologies for both, and at that time played a major role in the highly efficient and efficient use of large amounts of electronic information.

  Currently, a machine learning (ML) based approach is one of the main for automatic document classification. The optimal performance of a machine learning (ML) based approach is highly dependent on the large amount of manually labeled training data. However, if there are hundreds or thousands of categories, the data labeling task is cumbersome and expensive, especially in complex document classifications.

  A lot of research has been done to use unlabeled data to improve the accuracy of the training model. However, existing methods cannot handle cases where training data is useless. Furthermore, since the learning process is highly dependent on a small number of training samples, the classification results are easily biased by the training data. For this reason, sufficient performance is not obtained for an actual system.

  The research of the present invention is closely related to research on information retrieval and text data mining (particularly document classification) that has been widely studied in conventional research. Basically, general approaches to automatic document classification can be divided into three types: supervised document classification, semi-supervised document classification, and unsupervised document classification. Their implementation generally includes two basic steps: a classification learning step and a document classification step.

  The supervised document classification approach treats category names only as symbolic labels and assumes no additional knowledge about their meaning. In addition, exogenous knowledge can be used to support the construction of classifiers. In the learning phase, a general recursive procedure is used that automatically constructs a classifier for a category by observing the characteristics of a document set that has been manually classified (eg, by a field expert) in advance. Thereafter, in the document classification stage, the classifier obtains the characteristics of the new document for classification under the corresponding category. Various different methods for the inductive construction of document classifiers have been examined in previous studies. The most common methods include probabilistic classifiers, decision trees, neural nets, support vector machines (SVM) and regression methods. Since knowledge about accurate classification for documents is used to manage classifier learning, a large number of training samples, manually labeled for all categories, are required for accurate learning.

  In order to reduce the human effort for training data labeling (classification), semi-supervised document classification approaches for document classification involving a small number of labeled data are gaining more and more attention. They utilize both labeled training data samples and unlabeled training data samples. Unlabeled data is used to improve the poor performance of supervised learning, including insufficient training data. So far, the work on the semi-supervised document classification approach can be broadly classified into generative methods, discriminative methods, and self-learning methods.

  The generative method assumes that example documents are generated from identifiable mixture distributions (eg, Gaussian mixture models). With a large amount of unlabeled data, it is possible to identify unknown parameters of the mixed model. A typical method is an Expectation-Maximization (EM) algorithm. Along the same way, document clustering is utilized to use unclassified documents to improve text classification, where each cluster actually serves as a “pseudo-mixed model”. The clustering process can be applied to classified data and unclassified data by introducing new features extracted from those clusters into patterns in the classified data and unclassified data.

  Discriminative methods are devised from the idea that data that is not classified into various classes is separated with a large margin. Based on this assumption, Transductive SVM (TSVM) (Transductive Support Vector Machine) extends the standard support vector machine with unclassified data to minimize misclassification of specific documents, Try to maximize "unclassified data margin". A logistics regression model, a common form of SVM, is also employed for semi-supervised text classification. Recently, a series of new semi-supervised learning approaches have emerged from graph representation. Here, labeled instances are represented as vertices, and unlabeled instances are represented as edges that encode the similarity between instances.

  The self-learning method assumes that the high confidence prediction of the classifier itself is accurate. There are two representative methods derived from this assumption. Self-training and co-training. Self-training is achieved as follows: 1) A small amount of labeled documents are used for classifier training, 2) The resulting classifier is used to classify unclassified documents 3) A reliable set of newly labeled documents that are selected with high confidence in each iteration is used to iteratively retrain the classifier. During this iteration, the classifier uses its own high confidence prediction to learn. As a similar technique, Patent Document 1 (Japanese Patent Laid-Open No. 2002-133389) provides an acceleration mechanism that uses test data distribution to improve the accuracy of iterative learning with a small number of training data. Joint training is realized as follows. 1) First, the feature set is divided into two sufficient and conditional independent sets that are used to train the two classifiers respectively; 2) Each classifier is then unlabeled data And select some reliable samples to extend the training data of the other classifier; 3) Both classifiers are retrained with additional training samples and the process repeats.

  Unlike supervised and semi-supervised learning methods that use the knowledge contained in the document set for document classification, the so-called unsupervised approach mainly focuses on the knowledge contained in the category name concept for automatic document classification. Use. They do not generate training documents by hand, but rather use an initial predefined keyword list or a keyword that appears as a root in the category name and employs certain bootstrapping mechanisms. . An alternative solution is to generate a set of training sentences using the keyword list of each category by dividing the document into sentences. At the same time, the classified sentences are used for document classification.

JP 2002-133389 A

  However, there are still problems to be solved for the existing methods as follows.

  First, for a supervised approach, creating sufficient training data for a supervised approach is very expensive. A supervised document classification approach requires a large amount of training data available for each document set or problem domain. However, they are often difficult, expensive, and time consuming to obtain labeled data because they require the effort of experienced annotators. Especially for complex tasks or domains with hundreds or thousands of classifications.

  Next, document classification results of the semi-supervised approach tend to be biased by a small number of training data. The idea of semi-supervised learning is not only to learn from labeled training data, but also to use structural information in available unlabeled data. The issue of the effectiveness of training data is partly addressed. Since the labeled data is sparse, not only is the accuracy insufficient, but its robustness is a major problem for the application of these methods.

  Furthermore, for unsupervised approaches, their document classification results tend to be biased by predefined keyword lists. In the so-called unsupervised approach, category names or keyword lists for each category serve as the root for a bootstrap mechanism for automatic text classification. This approach relies heavily on the initial keyword list defined by humans, and since there is no bias control mechanism, the accuracy and robustness of the classification results are generally not sufficient. In addition, initial source words need to be collected manually, which is a more verbose and costly task for complex tasks.

  Finally, their adaptability and scalability are insufficient for supervised, semi-supervised or unsupervised approaches. A classifier trained through all three approaches above depends on the domain or document set. That is, if the document set or domain is changed, the classifier needs to be retrained. For supervised and semi-supervised approaches, this means that additional human effort is required to label a certain amount of documents as training data. For the so-called unsupervised approach, when a domain is changed, it is necessary to define an initial keyword list that is relevant to the corresponding category. In addition, additional learning effort is required for the modified domain or document collection.

  In view of the above problems, there is a need for a new method and system for automatic document classification that improves the accuracy and scalability of document classification, especially when labeled data is not available.

(Object of invention)
The present invention has been proposed in view of the aforementioned problems of existing document classification approaches in this technical field.

  In the present invention, a document classifier generation method is proposed for automatic document classification. Data distribution knowledge about the target document set, and semantic information encompassed by the category name, is used to improve the accuracy and scalability of document classification, particularly when training data is not useful.

  In general, a hybrid document classifier formation method mainly includes three steps: (1) initial training data generation, (2) iterative classifier learning, and (3) final classifier construction.

  First, in the initial training data generation, the initial training data is generated based on the semantic analysis of the category name using an external knowledge source. For example, in an embodiment, a profile-based method is designed for forming a classifier. Here, each category has a semantically related feature set that serves as a representative profile of the category. With initial classifiers, initial training data (labeled documents) with positive and negative samples are generated for classifier learning for the next iteration.

  Next, in the iterative classifier learning phase, the classifier classification results from the last iteration in each iteration are used to construct the training data for that iteration (labeled as labeled data). Select results classified with high confidence). A new classifier is then created from the updated training data (data that has even been labeled). Finally, the new classifier replaces the classifier from the last iteration and is used to classify the remaining documents. If all documents are classified, the iteration ends when the classifier set converges or when other termination conditions are met.

  In the final classifier formation process, after the iterative learning is completed, the classifier that most closely matches the clustering result is selected as the final classifier from all the resulting classifiers. Since the present invention assumes that there is no training data, it mainly uses maximum likelihood estimation as a solution for classifier selection.

  Training data selection (including initial training data generation and iterative training data generation in iterative learning) helps align clustering and classification results when a Bayesian model can be adopted during the machine learning process. It should be noted that based. Its purpose is to mitigate bias that can arise from category names, external knowledge sources or noise data in the iterative classifier learning process.

  The method of generating a classifier according to the present invention obtains a clustering result for an object set, generates a rough category classification result for the object set to obtain a rough classifier, and generates a final classifier. The rough category classification result is adjusted with the clustering result. According to a preferred form, rough classifiers can be obtained externally as training data (it can be obtained from outside as manually labeled training data, or domain-related category names with reference to external knowledge sources) Can be generated automatically by learning a classifier. Furthermore, according to one aspect, the rough classification result can be adjusted by matching the rough classification result to the previously obtained clustering result. This adjustment process can be realized in an iterative manner. By repeatedly updating the training data, it is possible to learn the intermediate classifiers, from which the best classifier that best matches the clustering results is selected as the final classifier .

  A system for generating a classifier according to the present invention includes an acquisition unit that acquires a clustering result for an object set, a rough categorization unit that generates a rough category classification result for an object set to acquire a rough classifier, Adjustment / generation means for adjusting the rough category classification result with the clustering result to generate a final classifier.

  In the present invention, matching analysis between clustering and classification results is performed and integrated not only in the initial training data formation process but also in the iterative classifier learning process. This controls the bias that can arise from the semantic analysis corresponding to the category name. It ensures improved accuracy of the final classification result as well as the resulting training data.

  Furthermore, the method according to the invention does not require an initial predefined keyword list for training data or document classification. Instead, semantic analysis of category names by existing external knowledge sources (including hidden semantic analysis for simultaneous appearance keyword extraction) is used for initial training data generation. Since existing external knowledge sources can cover multiple domains, even if the domain or document set is changed, the method of the present invention offers a large number of additional labeling efforts in addition to a significant reduction. It can be easily applied to various types of domain / document sets.

  In addition, the mechanisms provided for final classifier formation include an iterative classifier learning process, particularly for characteristic classifiers (eg, SVM (Support Vector Machine), Logistic regression). It is possible to mitigate the risk that bias is overly applied to the classifier by noise data at. Furthermore, it is an important contribution of the present invention to improving the accuracy of the final result of document classification.

1 is an overall block diagram of a document classification system 100 (the internal structure of the classifier generation subsystem 10 is shown in detail therein) according to the present invention. 3 is a flowchart showing an example of operation processing of the document classification system 100 shown in FIG. 1. FIG. 2 is a block diagram for illustrating an internal configuration example of an adjustment / generation unit 103 in the classifier generation subsystem 10 illustrated in FIG. 1. It is a block diagram which shows the internal structure of the implementation example 400A of the rough categorization means 102 in the classifier production | generation subsystem 10 shown in FIG. 1, In the implementation example 400A, the training data labeled manually acquired from the outside are shown. Used directly for classifier learning. It is a block diagram which shows the internal structure of the implementation example 400B of the rough categorization means 102 in the classifier generation subsystem 10 shown in FIG. 1, and training data is automatically generated for classifier learning in the implementation example 400B. Is done. FIG. 6 is a block diagram showing an internal configuration of a training data generation unit 401B shown in FIG. 5 when training data is automatically generated. It is a block diagram which shows the example of an internal structure of the classification | category part 504 in the training data generation unit shown in FIG. It is a flowchart which shows the example of the operation | movement process 700 of a training data generation unit in the case of producing | generating the training data shown in FIG. 6 automatically. FIG. 7 is a block diagram illustrating an example of an internal configuration of a training data generation unit 505 for generating training data based on the intermediate classification result illustrated in FIG. 6, and a clustering result for a document set is acquired for adjustment of an intermediate classification result Is done. 6 is a flowchart showing an operation process 900 of the adjustment / generation unit 103 for iterative classifier learning in the classifier generation subsystem 10 shown in FIG. 1 according to the present invention. FIG. 2 is a schematic block diagram of a computer system used to implement the present invention.

  The foregoing and other features and advantages of the present invention will become more apparent from the following description taken in conjunction with the accompanying drawings. It goes without saying that the scope of the invention is not limited to the examples or specific embodiments described herein.

  The foregoing and other features of the present invention will be more fully understood when the following description is read in conjunction with the accompanying drawings.

  The classification generation method and system according to the present invention can be applied to text filtering, document recommendation, search result clustering, web page search, web mining system, and the like.

  FIG. 1 is a block diagram showing the entire document classification system 100 according to the present invention. FIG. 1 shows the internal structure of the classifier generation subsystem 10 in detail. As shown in the figure, the document set received from the document base 105 is clustered into a number of groups in advance by the document clustering means 107, and the clustering result is stored in the clustering result base 104. The clustering results for the document set stored in the clustering result base 104 are later consumed by the classifier generation subsystem 10 according to the present invention, or other applications related to information retrieval. With regard to document clustering methods, many existing approaches that are well known to those skilled in the art can be used with the present invention. These are not the main features of the present invention and will not be described in detail here. Any document clustering method can be used to obtain the required document clustering results as long as it is readily available to those skilled in the art. For example, the classification generation subsystem 10 according to the present invention shown in FIG. 1 includes an acquisition unit 101, a rough categorization unit 102, and an adjustment / generation unit 103.

  FIG. 2 is a flowchart showing an example of an operation processing procedure of the document classification system 100 shown in FIG.

The processing procedure 200 shown in FIG. 2 starts from step 201, where the classification generation subsystem 10 acquires a document set to be classified from the document base 105.
The acquired document set is provided to the rough categorization means 102 for rough categorization in order to generate a rough categorization result (that is, a rough classifier) as shown in step 202. For example, any of the existing supervised document classification, semi-supervised document classification, or unsupervised document classification methods as described in Background Art can be applied to implement rough or categorized purposes. . In one embodiment, for example, as described below, a method of learning a classification including training data can be employed to generate a rough classification. Training data for learning the classifier according to different application requirements can be entered externally as manually labeled training data or refer to semantic information about category names from external knowledge sources Can be generated automatically. Details of the training data generation processing procedure will be described later.

  At the same time, in step 203, the acquisition unit 101 acquires a clustering result stored in advance for the same document set from the clustering result base 104. As is known to those skilled in the art, the clustering results reflect the data distribution underlying the document collection. For this reason, the clustering result is used to suppress a bias that may occur in the rough classification result. Both the rough classification result regarding the document set from the rough categorizing unit 102 and the clustering result acquired by the acquiring unit 101 are supplied to the adjusting / generating unit 103. Next, in step 204, the adjustment / generation unit 103 uses the clustering result from the acquisition unit 101 to adjust the rough classification result (that is, the rough classifier) from the rough categorization unit 102. As a result, a final classifier 106 is generated. The principle and processing procedure for adjusting the rough classification result using the clustering result will be described with reference to FIG. Furthermore, as will be described later, this idea of adjusting rough classification results by using clustering results generates a group of intermediate classifiers from which one optimal classifier is selected as the final classifier. It is possible to extend iterative methods. By such a method, it is possible to further improve the accuracy of document classification. A specific iterative process of classification learning will be described later. Thereafter, in step 205, the document set obtained in step 201 is supplied to the generated final classifier 106, which in turn places each document into at least one appropriate category. Classify. The final classification result of the document is stored in the document classification result base 108. Thereafter, the processing procedure 200 ends.

  FIG. 3 is a block diagram showing an example of the internal configuration of the adjustment / generation unit 103 of the classifier generation subsystem 10 shown in FIG. In this example, it is assumed that rough categorizing means 102 processes rough classification by a method based on a query. Rough classification results are expressed as a series of rank scores. The adjustment / generation unit 103 sets a matching model based on Bayesian inference in order to perform matching between the rough classification result and the clustering result. In this way, a more accurate classification result (that is, the final classifier 106) can be realized. The method for adjusting the rough classification result including the clustering result is not limited to the matching example based on the Bayesian inference model as shown in FIG. It is easy for those skilled in the art to understand that other adjustment methods can be applied to achieve the purpose of improving the accuracy of document classification as well.

  In the example shown in FIG. 3, the adjustment / generation unit 103 includes a prior probability calculation unit 301 and a matching unit 302.

その結果、Ｐ（ｃ_ｊ｜ｄ_ｉ）＝ｓ'（ｄ_ｉ、ｃ_ｊ）と見なすことができる。 Prior probability calculation section 301 needs to first calculate prior probabilities corresponding to rough category classification results.
As described above, assume that the rough categorization result is represented as a series of rank scores.
Let C be a category set. The document _{d i} ∈D and category _{c i} ∈ C, rank score _{s (d} i, _{c j)} indicates the possibility _{that d i} belongs to _{c j} implicitly. Therefore, the score is normalized by Equation 1.

As a result, it can be considered that P (c _j | d _i ) = s ′ (d _i , c _j ).

ここで、事前確率Ｐ（ｃ_ｊ｜ｄ_ｉ）はラフな分類結果から得る。
明らかに、基礎的統計を利用することにより可能性を以下のように計算することができる。

よって、最終の整合モデルは以下のように示すことができる。

Thereafter, in the matching unit 302, a matching model is set based on Bayesian inference. Let C ′ be a cluster set. If the clustering result indicates that the document d _i has been clustered into cluster c ′ _k εC ′, then the matching result is indicated by the posterior probability as follows:

Here, the prior probability P (c _j | d _i ) is obtained from the rough classification result.
Obviously, by using basic statistics, the possibility can be calculated as follows:

Therefore, the final matching model can be expressed as follows.

  According to the matching model as shown in Equation 5, the final classifier adjusted by the clustering result can be achieved. Compared to the rough classifier shown in Equation 1, the final classifier ensures improved accuracy of the final category classification result. Furthermore, the biases derived from category names and corresponding semantic analysis can be effectively controlled by the introduction of category classification result adjustment based on clustering results.

  Hereinafter, the internal configuration of the rough categorization unit 102 of the classifier generation subsystem 10 will be described in more detail with reference to FIGS. 4 and 5. As described above, in one embodiment, a rough classifier can be generated by employing a classification learning method having training data. The training data employed in the present invention is manually labeled training data input directly from the outside, or can be automatically generated by the system. 4 and 5 respectively show the generation of rough classifiers by training the training data when using manually labeled training data or when the system automatically generates training data 2 Provide one example. Of course, generation of rough classifiers is not limited to training data learning, and other classifier generation methods known by those skilled in the art can also be applied to the present invention.

  First, referring to FIG. 4, in this example, the rough categorizing means 102 includes a training data generation unit 401 </ b> A and a learning unit 402. The training data generation unit 401A obtains manually labeled training data from the outside and supplies it directly to the learning unit 402 for classification learning. The learning unit 402 is then used to learn a rough classifier. Since the processing procedure for learning the classifier together with the training data is a well-known technique in the technical field to which the present invention belongs, a detailed description thereof will be omitted here.

  Referring to FIG. 5, in this example, rough categorizing means 102 includes a training data generation unit 401 </ b> B and a learning unit 402. Regarding the difference between the training data generation unit 401B and the training data generation unit 401A, the training data generation unit 401B automatically generates training data by referring to the semantic information about the category names from the external knowledge source 404. It is to be. Thereafter, as shown in FIG. 4, the generated training data is supplied to a learning unit 402 for learning a classifier.

  Hereinafter, the principle and processing procedure of automatic generation of training data by the training data generation unit 401B shown in FIG. 5 will be described in more detail with reference to FIGS.

  First, as illustrated in FIG. 6, the training data generation unit 401B includes, for example, a category name acquisition unit 501, a word meaning ambiguity resolution unit 502, a keyword generation unit 503, a classification unit 504, and a training data generation unit 505. Furthermore, as shown in FIG. 6, in addition to the document base 105, the training data generation unit 401B is also connected to a category name base 403 and an external knowledge source 404 regarding category names for performing automatic generation of training data. Yes.

  Training data automatic generation processing 700 by the training data generation unit 401B shown in FIG. 6 will be described with reference to the flowchart of FIG.

  The processing procedure 700 starts from step 701. In step 701, the category name acquisition unit 501 acquires a predetermined category name related to a set of documents from the category name base 403. Since words in category names have various meanings in various cases, in step 702, the word meaning ambiguity resolution unit 502 first performs word meaning ambiguity resolution on the category names obtained with the assistance of the external knowledge source 404. . Thereafter, in step 703, the category name after word meaning ambiguity resolution is supplied to the keyword generation unit 503, where an appropriate keyword is generated based on the identified word meaning. Here, suitable keywords may include words that are likely to appear simultaneously with the category name. It can be identified by hidden semantic analysis. In addition, they contain a narrower term, synonym or synonym for the keyword that appears in the category name. They may be found by an external knowledge source 404.

Here, in order to facilitate understanding, examples of word meaning ambiguity resolution and synonym selection are shown.
The word “spam” can have two meanings in WordNet. That is, (meaning 1): canned meat made mainly from pork, and (meaning 2): unnecessary e-mail.
We need to distinguish between them in order to pick a synonym for “spam” for product profile classification. Thus, “spam + canned meat made primarily from pork” and “spam + unwanted email” can be used as two queries sent to a document set (ie, a product profile set).
Assume that there are 20 hits for the former query and 100 hits for the latter query. Since 100> 20, it can be determined that “spam” in the context of this classification task has meaning 2. Thereafter, a synonym of meaning 2 (ie, “junk email”) is selected.

Returning to FIG. 8, in step 704, the generated appropriate keywords are supplied to a classifier 504 for classifying a set of documents to obtain an intermediate classification result (ie, an intermediate classifier). Next, in step 705, the intermediate classification result is supplied to the training data generation unit 505 for generating necessary training data. Thereafter, the processing procedure 700 ends.

FIG. 7 shows an example of the internal configuration of the classification unit 504 in the training data generation unit shown in FIG. In this example, a profile-based filtering method is utilized to generate intermediate classification results. That is, in order to search a document set, a category name related keyword is used as a query. Also, the documents in the hit list are labeled as corresponding categories.
As shown in FIG. 7, in this example, the classification unit 504 includes a search unit 601 and a category labeling unit 602. Referring again to step 704 in FIG. 8, step 704 shows that it includes several sub-steps. First, in sub-step 7041, the search unit 601 receives a category name related keyword from the keyword generation unit 503, and uses the keyword as a representative profile for searching a set of documents. Thereafter, in step 7042, the hit list is sent to the category labeling unit 602 as the search result is. Labels can label all or some (eg, the first 200) documents in the hit list to corresponding categories to achieve document classification.

In general, only the document at the top of the hit list is selected to ensure that the document labeled is accurate with high confidence.
For example, for the product category “anti_spam”, “Spam + Junk email” is sent as a keyword related to a document set for search.
Here, “spam” is identified from the category name (ie, “anti_spam”). “Junk e-mail” is a synonym selected from WordNet.
Assuming there are 1000 results returned in the hit list, the top 200 items may be selected as a representative product summary for the “anti_spam” product.
The top 200 product summaries will likely hold all the necessary features that a person will use to determine if the product has an anti_spam function or if the product belongs to the “anti_spam” category.

  As described above, after obtaining the intermediate classification result (that is, the intermediate classifier), the intermediate classification result is supplied to the training data generation unit 505 for generating training data. Various methods for generating training data known to those skilled in the art can be applied to the present invention. However, in order to further improve the accuracy of document classification in the processing procedure for generating training data, intermediate classification results can also be adjusted by employing clustering results (for example, by using Bayesian inference models). . FIG. 9 shows an internal configuration example of the training data generation unit 505. In the training data generation unit 505, the clustering result regarding the document set is used to adjust the intermediate classification result.

  It can be understood that the block diagram of FIG. 9 is somewhat similar to the internal configuration of the adjustment / generation unit 103 shown in FIG. That is, in this embodiment, the training data generation unit 505 uses a method similar to the adjustment / generation unit 103 in FIG. 3 in order to adjust the intermediate classification result. For the details, the description regarding FIG. 3 can be referred to. Thereafter, the adjusted (matched) intermediate classification result is supplied to the training data selection unit 802 to select desired training data.

  The configuration and operation principle of the classifier generation subsystem 100 according to the present invention has been described with reference to FIGS. As described above, in order to further improve the accuracy of document classification, the processing procedure for adjusting the rough classification result together with the clustering result is performed in an iterative manner. A detailed processing procedure will be described below with reference to FIG.

First, in step 901, training data generated during a processing procedure for generating a rough classification result is acquired as initial training data. During each iteration cycle, a classification learning method (eg, a multinomial model based on NB (Naive Bayesian)) is utilized to learn a new intermediate classifier along with the training data (step 902). Thereafter, in step 903, the new classifier is used to classify the document base 105 document to obtain a new intermediate classification result. In step 904, it is determined whether the iteration termination condition is met. The iteration end condition can be determined in advance by the user. For example, if all of the intermediate classifiers generated during the iterative process converge gradually, it is possible to select that the state of the training data is stable as an iterative termination condition. Alternatively, the fact that all the documents in the document base 105 are classified into the corresponding categories can be used as the repetition end condition. If it is not determined in step 904 that the iteration end condition is satisfied (that is, “NO” in step 904), the processing procedure proceeds to step 905. In step 905, the intermediate classification results generated in the series of iterations are utilized for the next iteration cycle to generate new training data. Here, the method of generating new training data according to the intermediate classification result is similar to that of FIG. As described above, the intermediate classification result is matched with the clustering result based on the matching model (for example, Bayesian matching model).
The difference from the method of FIG. 9 is in the calculation of prior probabilities.
Certain special methods may be employed for document classification results from various classifiers. For example, when an NB classifier is employed, the prior probabilities are P (c _j | d _i ) for each pair of categories c _j and documents d _i returned directly from the classifier.

Using an NB classifier as an example, this iterative algorithm can be designed as follows.
(A) Input of initial training data T: C → Powerset (D) (ie, labeled document subset);
(B) learning of the NB classifier by T and using the learning result to obtain P (c | d) for each category document pair (c, d) ∈ C × D;
(C) For each (c, d) εC × D, if dεC ′ is within the clustering result, P ′ ( c | d) = P (c | d, c ′);
(D) Generate several new labeled documents for training data T ′: C → Powerset (D);
Here, for each c∈C, T ′ (c), P ′ (c | d) is the highest in the document set D-domain (T) (difference set between D and domain set of T). Contains documents.
(E) If T ′ = Φ, the iterative process is terminated; otherwise, T: = T + T ′, jump to step b), and start the next iteration.

The NB classifier is taken as an example, and the iterative process for learning the classifier in steps 901 to 905 as shown in FIG. 10 has been described in detail.
During the iterative learning process, each iteration cycle produces a classifier that can be represented by a posterior probability function of the category document pair P ′ (c | d). Of course, the classifier according to the present invention is not limited to the NB classifier. Other types of classifiers can obviously be applied to the present invention.

  Returning to FIG. 10, if it is determined in step 904 that the iteration end condition is satisfied (that is, “Yes” in step 904), the processing procedure proceeds to step 906. In step 906, the group of classifiers generated during the iterative process is saved. Then, in step 907, an optimal classifier is selected as the final classifier from this group of classifiers. Here, as a typical selection method of the optimum classifier, there is a method of selecting an appropriate one according to a given document set. During the iterative learning process, it seems to reduce the bias of training data with insufficient clustering results. Therefore, the clustering result can be used to evaluate and select the most appropriate classifier. As an example, Bayesian models are used to select the optimal classifier.

最大尤度法に基づいて、Ｐ（Ｃ’｜Ｆ_ｋ）を最大にする特定のＦ_ｋを見つけ出す。
言うまでもなく、お互いに独立した文書とすると、次のように表すことができる。

ここで、ｃ’（ｄ）が文書ｄが属するクラスターであり、ｃ（ｄ）はｄが分類器Ｆ_ｋに属するカテゴリである。

同様に、上述した整合モデルの尤度計算として、Ｆ_ｋの尤度関数は以下の通
りである。

そして、最終分類器は、

として導かれる。
For example, let the intermediate classifier be F _k , k = 1, 2,... (N). Here, N indicates the time of repetition.
With the following formula including the Bayesian model

Find the specific F _k that maximizes P (C ′ | F _k ) based on the maximum likelihood method.
Needless to say, if the documents are independent from each other, they can be expressed as follows.

Here, c '(d) is a cluster that belongs document d, c (d) is a category belonging to d is the classifier F _k.

Similarly, as the likelihood calculation of the above-described matching model, the likelihood function of F _k is as follows.

And the final classifier is

As led.

  Thereafter, when the final classifier is selected, the processing procedure 900 ends.

  FIG. 11 is a schematic block diagram of a computer system 1000 used to implement the present invention. As shown, the computer system 1000 includes a CPU 1001, a user interface 1002, a peripheral device 1003, a memory 1005, an external storage device 1006, and an internal bus 1004 that connects the above components to each other. The memory 1005 further includes a domain and POS analysis module, an automatic document classification module, a document clustering module, an IR related system, an operating system (OS), and the like. The present invention mainly relates to an automatic document classification module. This is, for example, the document classification system 100 shown in FIG. The document clustering module performs a clustering process on the document set, and stores the clustering result in an appropriate clustering result base (for example, the clustering result base 104). The external storage device 1006 stores various databases related to the present invention such as the clustering result base 104, the document base 105, the document classification result base 108, the category name base 403, the external knowledge source 404, and the like.

  The document classification method and system according to the present invention have been described with reference to the accompanying drawings. Based on the above description, the effects of the present invention will be described below.

  In the present invention, the matching analysis between the results of clustering and classification is performed and integrated not only in the initial training data formation process but also in the iterative class learning process. In this process, biases that may arise from category names and corresponding semantic analysis are controlled. It ensures an improvement in the accuracy of the final classification result as well as the resulting training data.

  Furthermore, the method according to the invention does not require training data for document classification or an initial predefined keyword list. Instead, semantic analysis of category names (including hidden semantic analysis for co-occurrence keyword extraction) with the aid of existing external knowledge sources is used for initial training data generation. Since existing external knowledge sources can cover multiple domains, when a domain or document collection is changed, the method of the present invention can be applied to a large number of different labels with significantly reduced additional labeling efforts. It can be easily applied to different types of domains / document sets.

  In addition, the mechanisms provided for final classifier formation are iterative classifier learning, especially for characteristic classifiers (eg, SVM (Support Vector Machine), Logistic regression). It is possible to reduce the risk that bias is overly applied to the classifier due to noise data in the process. Furthermore, it is an important contribution of the present invention to improving the accuracy of the final result of document classification.

  Specific embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific configurations and processes shown in the accompanying drawings. In the above embodiments, some specific steps are shown and described as specific examples. However, the method processing of the present invention is not limited to these specific steps. Those skilled in the art can change, modify and supplement these steps, or can change the order of several steps without departing from the spirit and essential function of the present invention. You will understand that.

  The elements of the invention can be implemented in hardware, software, firmware or a combination thereof and can be utilized in a system, subsystem, component or subcomponent. When implemented in software, the elements of the invention are programs or code segments for performing the necessary tasks. The program or code segment can be stored on a computer readable medium or transmitted by a data signal contained on a transmission cable or carrier wave on a communication link. Computer-readable media includes all media that can store or transfer information. Specific examples of computer-readable media are electronic circuits, semiconductor storage devices, ROM, flash memory, erasable ROM (EROM), flexible disks, CD-ROM optical disks, hard disks, optical fiber media, radio frequency (RF) links. Etc. The code segment can also be downloaded via a computer network such as the Internet or an intranet.

  Although the present invention has been described above with reference to specific embodiments, the present invention is not limited to the specific embodiments and specific configurations shown in the drawings. For example, some of the components shown may be combined with each other as one component. Alternatively, one component may be divided into several subcomponents and other known components may be added. The operation process is not limited to that shown in the embodiment. Those skilled in the art will appreciate that the present invention can be implemented in other specific forms without departing from the spirit and essential function of the invention. Accordingly, the current embodiment is to be considered in all respects as illustrative and not restrictive. The scope of the invention is indicated by the appended claims rather than by the foregoing description. Accordingly, all modifications that come within the meaning and range equivalent to the terms of the claims are included in the scope of the present invention.

100: Document classification system 10: Classifier generation subsystem 101: Acquisition unit 101
102: rough categorization means 103: adjustment / generation means 104: clustering result base 105: document base 107: document clustering means 108: document classification result base 106: final classifier 301: prior probability calculation unit 302: matching unit 401A: training Data generation unit 402: Learning unit 401B: Training data generation unit 402: Learning unit 403: Category name base 404: External knowledge source 501: Category name acquisition unit 502: Word meaning ambiguity resolution unit 503: Keyword generation unit 504: Classification unit 505 : Training data generation unit 504: classification unit 601: search unit 602: category labeling unit 505: training data generation unit 8011: prior probability calculation unit 8012: matching unit 80 : Training data selection unit 803: classifier

Claims (10)

A classifier generation method performed by a computer constituting a classifier generation system,
A rough categorization step that generates a rough categorization result for a set of objects to obtain a rough classifier;
Adjusting and generating the rough category classification result with the clustering result for the object set to generate a final classifier,
The rough categorization step comprises:
The rough category classification result is generated by any one of supervised document classification, semi-supervised document classification, and unsupervised document classification,
The adjustment / generation step includes
A prior probability calculating step of calculating a prior probability corresponding to the rough category classification result;
Matching the rough category classification results against the clustering results based on Bayesian inference using the prior probabilities, and generating the final classifier,
The final classifier is a Bayesian classifier in supervised document classification;
The rough categorization step includes:
A training data generation step for acquiring training data;
Learning the rough classifier from the training data,
The training data is
A category name obtaining step for obtaining a category name that is a name of a classification indicating the property of the object set;
A keyword generation step of generating a related keyword based on the category name;
A classification step for generating an intermediate category classification result from the keywords;
Automatically generated by a training data generation step of generating the training data from the intermediate categorization result,
The keyword is used as a representative profile,
The classification step includes
A search step using the representative profile as a query term to search the object set;
Labeling objects in the hit list as search results for corresponding categories,
The training data generation step includes
Calculating a prior probability corresponding to the intermediate categorization result;
Using the prior probabilities to match the intermediate category classification result with the clustering result based on Bayesian inference, and generating a classifier for use in training data generation. Generation method. The classifier generation method according to claim 1 , wherein the training data is training data manually labeled. The keyword generation step further includes:
Performing word sense disambiguation on the category name obtained with reference to an external knowledge source;
The method for generating a classifier according to claim 1, further comprising: generating the related keyword based on the category name after word meaning ambiguity resolution.   The classifier generation method according to claim 1, wherein a predetermined upper number of objects in the hit list are labeled with corresponding categories. In the adjustment / generation step,
Iterative classifier learning is performed by using the training data generated during the processing procedure for generating the rough category classification result and the rough category classifier as initial training data and initial classifier, respectively. Run,
In one cycle of iteration in the iterative classifier learning,
Learn the intermediate classifier corresponding to the current cycle of the iteration with the training data generated in the cycle before the iteration,
Classify a set of objects by using a learned intermediate classifier corresponding to the current cycle of iteration to obtain an intermediate categorization result of the current cycle of iteration;
Adjust the categorization results in the middle of the current cycle of iterations with the clustering results to generate training data used for the next cycle of iterations,
The classifier that most closely matches the clustering result is selected as a final classifier from the group of intermediate classifiers learned in each cycle when the iteration end condition is satisfied. A method of generating a classifier described in 1. A system for generating a classifier,
Rough categorization means for generating rough categorization results for object sets to obtain a rough classifier;
Adjusting / generating means for adjusting the rough category classification result with the clustering result for the object set to generate a final classifier;
The rough categorizing means is
The rough category classification result is generated by any one of supervised document classification, semi-supervised document classification, and unsupervised document classification,
The adjusting / generating means is
A prior probability calculating means for calculating a prior probability corresponding to the rough category classification result;
Matching means for matching the rough category classification result against the clustering result based on Bayesian inference using the prior probability and generating the final classifier;
The final classifier is a Bayesian classifier in supervised document classification;
The rough categorizing means is
A training data generation unit for acquiring training data;
A learning unit that learns the rough classifier from the training data,
A category name base for storing a domain related to a category name which is a name of a classification indicating properties of the object set;
The training data generation unit is
A category name acquisition unit for acquiring the category name related to the object set;
A keyword generating unit that generates a related keyword based on the category name;
A classification unit for generating an intermediate category classification result from the keyword;
A training data generation unit for acquiring the training data from the intermediate category classification result, automatically generating the training data,
The keyword is used as a representative profile,
The classification unit includes:
A search unit that uses the representative profile as a query term to search the object set;
A category labeling unit that labels objects in the hit list as a search result for the corresponding category,
The training data generation unit
A unit for calculating a prior probability corresponding to the intermediate category classification result;
A classifier comprising: a unit that generates a classifier for use in training data generation by matching the intermediate category classification result with the clustering result based on Bayesian inference using the prior probability. Generation system.   The classifier generation system according to claim 6, wherein the training data generation unit obtains training data manually labeled from the outside. An external knowledge source for storing knowledge about the category name;
The training data generation unit includes a word sense ambiguity resolution unit that performs word sense ambiguity resolution for a category name acquired with reference to the external knowledge source,
The classifier generation system according to claim 6, wherein the training data generation unit generates a related keyword based on a category name after word meaning ambiguity resolution.   7. The classifier generation system according to claim 6, wherein the category labeling unit levels a predetermined number of higher-order objects in the hit list to corresponding categories. The adjusting / generating means includes
Iterative classifier learning is performed by using the training data generated during the processing procedure for generating the rough category classification result and the rough category classifier as initial training data and initial classifier, respectively. And
In one cycle of iteration in the iterative classifier learning,
Learn the intermediate classifier corresponding to the current cycle of the iteration with the training data generated in the cycle before the iteration,
Classify a set of objects by using a learned intermediate classifier corresponding to the current cycle of iteration to obtain an intermediate categorization result of the current cycle of iteration;
Adjust the categorization results in the middle of the current cycle of iterations with the clustering results to generate training data used for the next cycle of iterations,
The classifier that most closely matches the clustering result is selected as a final classifier from the group of intermediate classifiers learned in each cycle when the iteration end condition is satisfied. The classifier generation system described in 1.

Images