KR20120059935A - Text classification device and classification method thereof - Google Patents

Abstract

PURPOSE: A document classification apparatus and document classification method thereof are provided to calculate weighted values for documents of learning data based on information related to a category of the learning data. CONSTITUTION: Weighted values for learning data are calculated according to similarity between execution data and the learning data(S120). Document classification learning operation for the learning data is executed by reflecting the weighted values for the learning data(S130). The execution data is classified based on results of the document classification learning operation(S150). The learning data includes plural documents classified by each category. The documents include different weighted values according to the similarity.

Description

TEXT CLASSIFICATION DEVICE AND CLASSIFICATION METHOD THEREOF

The present invention relates to a document classification apparatus and a document classification method thereof, and more particularly, to a machine learning based document classification apparatus and a document classification method thereof.

There are tens and hundreds of millions of documents on the Internet, and the volume of documents is increasing exponentially due to the activation of blogs and mini homepages. These documents have a lot of information, and various search systems and analysis systems are used to access the information contained in the documents.

Most retrieval and analysis systems for accessing document information improve accessibility by dividing documents into categories. For example, in the case of a portal search system that provides Internet news, documents are classified into categories such as politics, society, economy, and entertainment to enhance access to documents. Initially, the classification of these documents was done directly by humans. However, as the amount of information increases, there is an increasing need for a document classification apparatus that can automatically classify many documents.

SUMMARY OF THE INVENTION The present invention provides a document classification apparatus for automatically classifying documents and simultaneously classifying documents in each category according to information included in the documents.

According to an embodiment of the present invention, a document classification method for classifying documents by category may include: calculating weights for the learning data according to a degree of similarity between execution data and learning data; Performing a document classification learning operation on the learning data by reflecting the weighted values of the learning data; And classifying the execution data based on a result of performing the document classification learning operation.

In an embodiment, the learning data includes a plurality of documents classified for each category, and the plurality of documents of the learning data have different weights according to similarities with the execution data.

In an embodiment, the degree of similarity between the plurality of documents of the learning data and the execution data may be determined by the occurrence probability of predetermined words included in the plurality of documents of the learning data and the occurrence probability of predetermined words included in the execution data. Is determined accordingly.

In an embodiment, the performing of the document classification learning operation may include calculating a probability of occurrence (hereinafter, learning data distribution) of predetermined words included in the learning data by reflecting a weight of the calculated learning data; And determining whether the learning data distribution is similar to a probability of occurrence of certain words of the execution data (hereinafter, referred to as execution data distribution).

In an embodiment, the degree of similarity between the learning data distribution and the execution data distribution is determined depending on whether the values of likelihoods for the execution data distribution converge.

In an embodiment, when the values of the likelihood for the execution data distribution do not converge, after the execution data is classified, the weight for the learning data including the classified execution data is determined again.

Document classification apparatus for classifying documents by category according to an embodiment of the present invention includes a weight calculation module for calculating a weight for the documents of the training data; A document classification learning module for calculating a probability of occurrence of words included in the learning data (hereinafter, referred to as learning data distribution) by reflecting the weights; And a document classifier for classifying execution data based on the learning data distribution reflecting the weight calculated by the document classification learning module.

In an embodiment, the weight calculation module calculates weights for documents of learning data according to a similarity degree of occurrence probability of predetermined words of the learning data and the execution data.

In an embodiment, the weights of the documents of the learning data may be different from each other.

According to an embodiment, the document classification learning module may include the execution data in the category of the learning data when the learning data distribution reflecting the weight is not similar to the occurrence probability of the words of the execution data (hereinafter, referred to as execution data distribution). After classifying temporarily, the weights for the documents of the learning data are again determined.

In an embodiment, the degree of similarity between the learning data distribution and the execution data distribution is determined depending on whether the values of the likelihoods for the execution data distribution converge.

In an embodiment, when the value of the likelihood for the distribution of execution data does not converge, the document classification learning module temporarily classifies the execution data, and assigns to the category of the learning data including the temporarily classified execution data. Information about the weight calculation module.

In an embodiment, the weight calculation module recalculates the weights of the documents of the learning data based on the information about the category of the learning data including the temporarily classified execution data.

According to an embodiment of the present disclosure, documents may be classified according to categories.

1 to 3 are views for explaining the operation of the document classifier according to an embodiment of the present invention.
4 is a block diagram showing a document classification apparatus according to another embodiment of the present invention.
5 is a diagram illustrating an exemplary algorithm for performing an operation of the document classification apparatus of FIG. 4.
6 is a flowchart for explaining an operation of the document classification apparatus of FIG. 4.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described with reference to the accompanying drawings so that those skilled in the art may easily implement the technical idea of the present invention.

1 to 3 are diagrams for explaining the operation of the document classifier 10 according to an embodiment of the present invention.

Referring to FIG. 1, the document classifier 10 receives a document to be classified, and classifies the received document into a category corresponding to the received document. For example, the document classifier 10 may classify the received document x as category C based on the content described in document x.

In this case, the document classifier 10 may classify the received document based on the machine learning method. Here, the machine learning method means finding a common pattern of already classified documents. The document classification method based on the machine learning method can automatically classify the received document by applying the common pattern of already classified documents to the document to be classified.

2 and 3, the data is collected (step S11), and the learning data is generated by classifying the collected data into categories (step S12). Here, the learning data means a set of documents classified by each category. For example, the initial learning data may be a set of documents classified by each category by a person. Thereafter, a common pattern of documents classified for each category is extracted (step S13).

When the execution data is received in the document classifier 10 (step S21), the document classifier 10 compares the common pattern of the documents classified for each category with the execution data (step S22). Here, execution data means a collection of documents to be classified. The document classifier 10 may mechanically classify the execution data by comparing the extracted common pattern with the execution data (step S23).

In this case, for example, the common pattern extracted for each category may be a frequency of occurrence of a specific word. For the sake of brevity, assume that words such as 'baseball' and 'soccer' appear in a certain category in the category 'sports'. The document classifier 10 may classify documents of execution data in which these words appear at similar frequencies based on the frequency of occurrence of words such as 'baseball' and 'soccer' into the 'sports' category.

However, in this case, the distribution of the execution data to be classified by the document classifier 10 may not match the distribution of the learning data. This is because the types of words appearing in the document may be infinite due to new words, foreign words, and the like. Here, the distribution of the execution data means a probability distribution for the frequency of occurrence of specific words in the document of the execution data, and the distribution of the training data means a probability distribution for the frequency of occurrence of specific words in the learning data.

In order to solve such a problem, the document classification apparatus according to another embodiment of the present invention will weight each document in the learning data according to the similarity between the document of the execution data and the documents of the learning data. For example, the document classification apparatus may give a high weight to a document similar to the document of the execution data among the documents in the learning data, and give a low weight to a document that is not similar to the document of the execution data.

In this case, the similarity between the documents in the learning data and the document of the execution data may be determined according to, for example, the probability of occurrence of a predetermined word. By assigning a weight, the document classification apparatus according to the embodiment of the present invention may classify execution data more precisely into each category. This will be explained in more detail below.

4 is a block diagram showing a document classification apparatus 100 according to another embodiment of the present invention. Referring to FIG. 4, the document classification apparatus 100 includes a document classifier 110 and a document classification learner 120. The document classification learner 120 includes a weight calculation module 121 and a document classification learning module 122.

The weight calculation module 121 receives the execution data and the learning data. The weight calculation module 121 measures the similarity between the execution data and the learning data, and gives a high weight to the document in the learning data similar to the document of the execution data among the documents in the learning data. In this case, the weight may be set higher as the probability that a specific word in the document in the learning data has a similarity in the distribution of the execution data and the probability of the execution data distribution. The weight may be calculated using Equations 1 to 5 below.

Equation 1 is a function for determining the weight of the document 'd' of the training data using the similarity function K () between the document 'd' of the training data and the document 'du' (du∈Du (cj)) of the execution data. Shows 'Φ'. At this time, Du (cj) means a document belonging to the category cj among the execution data, adu is a weight of the document 'du' of the execution data, it means the importance of the document 'du' of the execution data in the execution data distribution. . In this case, adu may be calculated using Equation 2.

In Equation 2, P1 denotes a distribution of learning data, and P2 denotes a distribution of execution data. π is the weight of the document of the calculated learning data. In Equation 2, a 'is determined to minimize the KL between the distribution of the execution data and the distribution of the learning data. KL stands for Kullback-Leibler divergence. KL of Equation 2 may be expressed as Equation 3.

Using Equation 3, it is possible to minimize the KL between the distribution of the training data and the distribution of the execution data. This may be expressed as in Equation 4.

In Equation 4, Ncj means the number of documents belonging to the cj category among the documents of the execution data. For convenience of calculation, it is assumed that the total sum of the weights is equal to the number of documents. In this case, Equation 4 may be formulated as in Equation 5, and an optimized weight according to an embodiment of the present invention may be calculated using Equation 5.

4, the document classification learning module 122 receives the weights of the documents in the learning data from the weight calculation module 121. In addition, the document classification learning module 122 receives the learning data and the execution data. The document classification learning module 122 performs a document classification learning operation by using the weights of the documents in the received learning data.

Here, the document classification learning operation includes an operation of calculating a probability value for the frequency of occurrence of words appearing in each document of the learning data. The document classification learning module 122 may calculate the probability value by reflecting the weights in the documents in the learning data, thereby more importantly reflecting the words appearing in the high weight document in the calculation of the probability value. That is, the document classification learning module 122 may obtain a distribution of learning data in which weights are reflected. In this case, the document classification learning module 122 may perform a document classification learning operation by using a well-known Naive Bayes-based document classifier.

When performing a document classification learning operation using a Naive Bayes-based document classifier, the document classification learning module 122 performs the document classification learning operation to maximize the likelihood. When the likelihood converges to a predetermined maximum value, the document classification learning module 122 may transmit the calculated probability value (or learning data distribution) to the document classifier 110. In this case, the document classifier 110 may classify the documents in the execution data into categories by comparing the received probability value (or learning data distribution) with the execution data (or execution data distribution).

On the other hand, if the likelihood does not converge to a predetermined maximum value, the document classification learning module 122 temporarily classifies the execution data, and transmits the category information of the temporarily classified execution data to the weight calculation module 121. . The weight calculation module 121 reflects the category information of the received temporary execution data in the training data, and determines the weight of the documents in the training data again. In the document classification learning module 122, the likelihood may be expressed as in Equation 6.

In Equation 6, Dl means learning data, Du means execution data, and θ means a parameter to be learned. For convenience of explanation, it is assumed that P2 (θ) is uniform. In this case, the likelihood to converge to the maximum value may be expressed as in Equation 7.

In Equation 7, Z denotes 0 or 1 whether or not the document d belongs to the j-th category. y is a set (or category label) of the category of the document d, and is given through the training data D1. P2 means distribution of execution data, and P1 means distribution of learning data. By way of example, it is assumed that P1 and P2 have different values. In this case, the classifier (eg, Naive Bayes based document classifier, or the document classifier of FIG. 4) aims to maximize the likelihood at P2. In Equation 7, the likelihood at P1 is calculated and reflected because the likelihood at P1 calculated by reflecting the weight? Is the likelihood at P2. This is because the weight π makes P1 and P2 as equal as possible.

To maximize the likelihood, an EM algorithm based semi-supervised learning method will be applied. The EM algorithm is divided into two stages such as Equation 8 and Equation 9.

Using the values obtained in Equation 8, the value of the parameter θ can be obtained as in Equation 9.

In Equation 9, n (w, d) means the number of times the word w appears in the document d. D and W may be calculated as shown in Equation 10.

As described above, the document classification learner 120 according to an embodiment of the present invention reflects the weights of the documents in the learning data when calculating the learning data distribution. By using the weights of the documents in the learning data, the document classification apparatus 100 according to an embodiment of the present invention can stably classify the execution data even when the distribution of the execution data and the distribution of the learning data are different. Document classification learning operation using the weight π described in Equations 6 to 10 may be implemented through the algorithm shown in FIG. 5.

Meanwhile, the document classifier 110 receives information regarding execution data and learning data distribution. The document classifier 110 classifies the execution data using the information on the distribution of the learning data. Here, the document classifier 110 may be implemented using the document classifier 10 described with reference to FIGS. 1 to 3.

6 is a flowchart for describing an operation of the document classification apparatus 100 of FIG. 4.

In operation S110, execution data is received by the document classification apparatus 100. Execution data includes at least one document to be classified.

In step S120, the weights of the documents in the learning data are determined. That is, the weight calculation module 121 determines weights of documents in the learning data based on the similarity between the execution data and the learning data. For example, a document of learning data similar to a document to be classified may be given a higher weight than a document of dissimilar learning data.

In step S130, the document classification learning operation is performed. In this case, the document classification learner 122 reflects the weights of the documents of the learning data calculated by the weight calculation module 121 and calculates the distribution of the learning data.

In step S140, it is determined whether the likelihood converges to a predetermined maximum value. That is, the document classification learner 122 calculates the likelihood and determines whether the calculated likelihood value converges to a predetermined value.

If the likelihood does not converge to a predetermined maximum value, the document classification learner 122 temporarily classifies the execution data (step S150). The category information of the temporarily classified execution data is transferred to the weight calculation module 121, and the weight calculation module 121 determines the weight again by reflecting the category information of the temporary execution data (S160).

When the likelihood converges to a predetermined maximum value, the document classification learner 122 transmits the information about the calculated learning data distribution to the document classifier 110. The document classifier 110 finally classifies the execution data by using the received information about the distribution of the learning data (step S170).

As described above, the document classification apparatus 100 according to the embodiment of the present invention assigns weights to the documents of the training data, respectively, so that the execution data can be accurately displayed even when the distribution of the training data and the distribution of the execution data are not the same. Can be classified.

On the other hand, it is apparent to those skilled in the art that the structure of the present invention can be variously modified or changed without departing from the scope or technical spirit of the present invention. In view of the foregoing, it is believed that the present invention includes modifications and variations of this invention provided they come within the scope of the following claims and their equivalents.

10, 110: document sorter 100: document sorting device
120: classifier learner 121: weight calculation module
122: Document Classification Learning Module

Claims (13)

In the document classification method to classify documents by category:
Calculating weights for the learning data according to a degree of similarity between the execution data and the learning data;
Performing a document classification learning operation on the learning data by reflecting the weight on the learning data; And
And classifying the execution data based on a result of performing the document classification learning operation. The method of claim 1,
The learning data includes a plurality of documents classified for each category, and the plurality of documents of the learning data have different weights according to similarities with the execution data. The method of claim 2,
The degree of similarity between the plurality of documents of the learning data and the execution data is a document determined according to the occurrence probability of predetermined words included in the plurality of documents of the learning data and the occurrence probability of predetermined words included in the execution data. Classification method. The method of claim 1,
Performing the document classification learning operation is
Calculating a distribution of learning data by reflecting a weight of the learning data; And
And determining whether the learning data distribution is similar to the execution data distribution. The method of claim 4, wherein
A degree of similarity between the learning data distribution and the execution data distribution is determined according to whether or not values of likelihood for the execution data distribution converge. The method of claim 5, wherein
And if the values of the likelihoods for the distribution of execution data do not converge, after the execution data is classified, the weight for the learning data including the classified execution data is determined again. In a document classification device that classifies documents by category:
A weight calculation module for calculating weights for documents of the learning data;
A document classification learning module for calculating a distribution of learning data by reflecting the weights; And
And a document classifier for classifying execution data based on the learning data distribution calculated by the document classification learning module. The method of claim 7, wherein
And the weight calculation module calculates weights for documents of learning data according to a similarity degree of occurrence probability of predetermined words of the learning data and the execution data. The method of claim 8,
Document classification apparatus, characterized in that the weights for the documents of the learning data are different from each other. The method of claim 7, wherein
The document classification learning module temporarily classifies the execution data into a category of the learning data when the distribution of the learning data is not similar to the distribution of the execution data, and then re-determines the weights of the documents of the learning data. Sorting device. 11. The method of claim 10,
A degree of similarity between the learning data distribution and the execution data distribution is determined according to whether the values of the likelihoods for the execution data distribution converge. The method of claim 11,
If the value of the likelihood for the distribution of execution data does not converge, the document classification learning module temporarily classifies the execution data and provides information on a category of learning data including the temporarily classified execution data. Document classification apparatus for transmitting to the weight calculation module. The method of claim 12,
And the weight calculation module is configured to recalculate weights for documents of the learning data based on the information on the category of the learning data including the temporarily classified execution data.

Images