KR100876319B1 - Apparatus for providing document clustering using re-weighted term - Google Patents

Abstract

A clustering apparatus and a method thereof are presented to minimize semantic difference between a cluster required by a user and a cluster provided by a system. According to a document clustering apparatus using a terminology weight recalculation, inputted document information is dissembled into each sentence information and invalid words are removed and primitive is extracted. A pre-processor(200) clusters documents on the basis of non-negative number matrix factorization, by generating a terminology-sentence matrix. A document clustering part(300) recalculates weight of the terminology and generates document clusters.

Description

Apparatus for providing Document Clustering using Re-weighted Term}

The present invention relates to a document clustering apparatus using term weight recalculation, and more particularly, to re-calculate a term weighted by user feedback in order to minimize the semantic difference between higher level user demand and lower level system cluster. The present invention relates to a document clustering apparatus using term weight recalculation that can improve the performance of a document clustering method by using a semantic feature matrix and a semantic variable matrix representing an internal structure of a document set in a document clustering process.

Document clustering is the discovery of groups of documents with similar characteristics from a set of documents by a clustering algorithm. Document clustering is an important document analysis method used in many applications of information retrieval, such as organizing data, browsing web search results, or multiple document summaries.

In document clustering, the distribution of data sets, internal structure, or the type of clustering desired by users have a significant effect on clustering results. There have been studies to improve the quality of clustering by providing limited information to users.

As one of these studies, a semi-supervised document clustering model has been proposed that integrates prior knowledge of cluster membership in document clustering analysis.

The quasi-map document cluster model encodes a cluster that a user wants to group, and adds the prior knowledge of the encoded user to a cost function of the cluster as a penalty term.

Another study is the quasi-K-means method, which creates initial seed clusters using labeled data and constrains them from labeled data. Create and cluster items.

In addition, a method of converting a cluster of web search results into a supervised learning method has been proposed. This method calculates several attributes from a ranked list of given queries and search results, and applies these attributes and training data to regression model training for the purpose of user browsing.

In another study, there is a method of classifying the characteristics of documents required for clustering from general characteristics not related to cluster names, and providing a means for adjusting parameters by applying various types of user feedback to the proposed model.

However, the above-described conventional methods have a problem in that the user feedback is not reflected or insufficient, causing a meaningful difference between the clustering result desired by the user and the clustering result by the system.

In addition, the above-described conventional methods have a problem in that it is difficult to reflect the internal structure of the document set in the document cluster, thereby ensuring the accuracy of the document cluster.

An object of the present invention is to provide a clustering device and a method for minimizing the semantic difference between a cluster required by a user and a cluster provided by a system.

Another problem to be solved by the present invention is to provide a clustering device and a method which can easily grasp the distribution of the internal structure and semantic features of the cluster.

Another object of the present invention is to provide a clustering apparatus and method for improving the accuracy of document clustering by recalculating the weights of terms.

Another problem to be solved by the present invention is to provide a term weight estimation method that can be applied to the conventional document clustering method to improve the performance of the clustering method.

(상기 수학식에서, J는 목적 함수이고, A는 용어-문장행렬로서

이며, W 는 비음수 의미특징 행렬로서 W=[w_ij]이고, H^T는 비음수 의미변수 행렬로서 H=[h_ij]이다.)The present invention relates to a document clustering apparatus using term weight recalculation, and as a previous step of document clustering using term weight recalculation, after decomposing input document information into respective sentence information, removing stop words, A pre-processing unit for extracting the terminology and generating the term-sentence matrix to enable clustering of documents based on non-negative matrix factorization (NMF); And a document cluster unit for factoring the term-sentence matrix with a non-negative matrix, recalculating the weights of terms, and generating a document cluster. Including but, the non-negative matrix factorization is characterized in that it is performed so that the objective function according to the following equation has a minimum value.

Where J is the objective function and A is the term-statement matrix

W is a non-negative semantic matrix, W = [w _ij ], and H ^T is a non-negative semantic matrix, H = [h _ij ].)

Preferably, the input unit for receiving document information for re-calculation of the term weight from the user, and classifies the received document; And an output unit for outputting the document clustering result generated by the document clustering unit for the user to check. It characterized in that it further comprises.

Also preferably, the preprocessing unit may include sentence decomposition means for decomposing the received document information into respective sentence information; Document cluster setting means for setting the number of document clusters in the received document information; Stopword removal means for removing stopwords from each of the decomposed sentences; Root extracting means for extracting roots from each sentence from which stop words are removed; Term-frequency vector generating means for generating a term-frequency vector according to the frequency of use of the term in each sentence from which stop words are removed; Term-sentence matrix generating means for generating a term-sentence matrix from the received document information; Characterized in that it comprises a.

Also preferably, the document clustering factor may factor the term-statement matrix into a non-negative semantic feature matrix (NSFM) and a non-negative semantic variable matrix (NSVM). Non-negative matrix factorization means; Non-negative matrix calculation means for calculating the non-negative semantic feature matrix and the non-negative semantic variable matrix; Weight recalculation means for recalculating a weight of terms using the non-negative semantic feature matrix and the non-negative semantic variable matrix; Document group generation means for grouping documents using the recalculated term weightings; Characterized in that it comprises a.

Also preferably, the stopword removal may be performed through a list of Rijsbergen stopwords.

delete

delete

delete

Also preferably, in order to update the element values of W and H ^T , repeating the calculation by the following equation until the objective function value is smaller than a preset convergence tolerance or exceeds a preset number of repetitions. It features.

Also preferably, the convergence tolerance is 0.0001 to 0.01.

Also preferably, the number of repetitions is 10 to 100.

Also, preferably, as a result of the non-negative matrix factorization, a non-negative semantic feature matrix and a non-negative semantic variable matrix are generated, and the non-negative semantic feature matrix and the non-negative semantic variable matrix generated by the following equation It is characterized by normalizing.

(In the above equation, w _ij represents an element of the non-negative semantic matrix, and h _ij represents an element of the non-negative semantic matrix.)

이면 문서 d_i를 군집 x에 할당하는 것을 특징으로 한다.Also preferably,

In this case, the document d _i is assigned to the cluster x.

Also preferably, the weight recalculation of the term is characterized by the following equation.

(A change in the average weight for the entire elements in the equation, Δg _a is a line, Δg _a ⁱ is the change amount of the i-th document in a second term weight, n is the number of the document. Also, g _a is is a weighted value of the a-th term, A _ai is the frequency of the a-th term and the i-th document's term, W and H are the non-negative semantic feature matrix and the non-negative semantic matrix, respectively, generated by factoring the non-negative matrix, I _i is the set of k other than ΔH _ki = 0 in the i-th document H _{* i} of H, H _ki ^old is the original value, and H _ki ^new is the value modified by the learning document.)

Also preferably, the initial value of g _a ^old may be set to 1.

Also preferably, H _ki ^new is characterized by being calculated by the following equation.

은 c번째 군집 내에서 의미특징의 중요도의 차이를 나타내고, ξ는 조절 상수이다.)(In the above equation, c is the serial number of the cluster, c = 1, 2, ..., e, e represents the number of clusters,

Is the difference in importance of the semantic features within the c th cluster, and ξ is the control constant.)

Also preferably, the adjustment constant ξ is 0.5.

Also preferably, the weight recalculation of the term is characterized by the following equation.

는 재계산된 가중치가 적용된 용어-문장행렬이며, G는 재계산된 용어의 가중치 행렬이다.)(Wherein A is a term-sentence matrix,

Is the term-sentence matrix with the recalculated weights applied, and G is the weight matrix of the recalculated terms.)

행렬에 K-평균(K-means) 방법을 이용하여 상기 문서 군집의 생성을 수행하는 것을 특징으로 한다.And preferably,

The generation of the document cluster is performed by using a K-means method on a matrix.

delete

delete

delete

delete

delete

According to the present invention, there is an effect of minimizing a semantic difference between a cluster desired by a user and a cluster by a system by using a document previously classified by the user.

According to the present invention, it is possible to easily grasp the distribution of the internal structure and semantic features of a cluster by using semantic features and semantic variables.

According to the present invention, the accuracy of document clusters can be improved by easily recalculating the weights of terms using semantic features and semantic variables.

According to the present invention, the weight estimation can be calculated by applying Singular Value Decomposition (SVD) as well as a document clustering method based on non-negative matrix factorization, thereby extending the application range.

According to the present invention, the weight of the estimated term can also be applied to the conventional document clustering method, thereby improving the performance of the conventional document clustering method.

Before describing the details for carrying out the present invention, it should be noted that the configuration that is not directly related to the technical gist of the present invention has been omitted within the scope of not distracting the technical gist of the present invention.

In addition, the terms or words used in the present specification and claims are consistent with the technical spirit of the present invention on the basis of the principle that the inventor can appropriately define the concept of the term in order to explain the invention in the best way. It should be interpreted as meaning and concept.

The present invention provides an apparatus for estimating a weight of terms based on non-negative matrix factorization (NMF) using a supervised learning method, and clustering documents by applying the estimated weights of the term to a clustered target set; Suggest a method.

The non-negative matrix factorization focuses on recognizing that a human recognizes an object as a combination of partial information of the object, and decomposes the object information into a base feature and an encoding feature, thereby removing the partial information. -Base).

The method of representing the entire object by the combination of such partial information can efficiently represent a large amount of information, and in a preferred embodiment of the present invention, a positive argument is found from a given positive matrix.

Hereinafter, a document clustering apparatus using term weight recalculation according to a preferred embodiment of the present invention will be described with reference to FIG. 1.

1 is an overall configuration diagram of a document clustering apparatus using term weight recalculation according to a preferred embodiment of the present invention.

As shown in FIG. 1, the document clustering apparatus using the term weight recalculation according to the preferred embodiment of the present invention includes an input unit 100, a preprocessor 200, a document cluster unit 300, and an output unit 400. It includes.

The input unit 100 receives document information for recalculating a weight of a term from a user side, and classifies the received document to correspond to a cluster label.

In addition, the preprocessing unit 200 is a previous step of document clustering by recalculating the weights of terms, decomposing the input document information into respective sentence information, removing unnecessary strings, extracting roots, and non-negative matrices. Performing a function to cluster the documents based on the factorization, sentence decomposition means 210, document cluster setting means 220, stop word removal means 230, root extracting means 240 , Term-frequency vector generating means 250, term-phrase matrix generating means 260.

First, the sentence decomposing means 210 decomposes the received document information into respective sentence information.

Next, the document group setting means 220 performs a function of setting the number of document clusters in the received document information. In this embodiment, the number of document clusters is set to k, but the present invention is limited thereto. Not.

Next, the stopword removing means 230 performs a function of removing stopwords through a stopword list (for example, Rizbergen-rijsbergen, etc.) for each sentence decomposed through the sentence decomposition means 210.

For reference, stopwords (Stop-words or Noise Word) refers to words that are ignored at the time of search or words that search engines exclude from the index when building the database. Verbs, including, in, surveys, suffixes, conjunctions, endings, etc., and in English, including a, an, is, are, if, for, but, this, may, There are modal verbs, personal pronouns, instructional pronouns, and prepositions.

Next, the root extracting means 240 performs a function of extracting the root for each sentence.

For reference, the root is a morpheme representing the actual meaning of the word among the morphemes constituting the word, and is a real real morpheme synthesized in the case of a compound word, and the rest except the affix in the case of a derivative word. For example, the roots include the cover, the cover, the person, the person, and the generous.

Next, the term-frequency vector generating means 250 performs a function for generating a term-frequency vector for each sentence according to the frequency of use of the term except the stopword.

Finally, the term-sentence matrix generating means 260 performs a function of generating a matrix A consisting of m × n composed of m terms and n sentences in the document.

In this case, the matrix A may be represented by A = [A _{* 1} , A _{* 2} , ..., A _{* n} ], and each row vector A _{* i} is a term-frequency vector of the i-th document.

In addition, the document grouping unit 300 performs a function of recalculating the weights of terms and generating document clusters by non-negative matrix factorization of the matrix A generated through the term-sentence matrix generating unit 260. The non-negative matrix factorization means 310, the non-negative matrix calculation means 320, the weight recalculation means 330, and the document cluster generation means 340 are included.

The non-negative matrix factoring means 310 factorizes the matrix A using a non-negative semantic feature matrix (NSFM) and a non-negative semantic variable matrix (NSVM). It performs the function.

Hereinafter, the decomposition of the term-statement matrix using the non-negative matrix factorization (NMF) will be described.

Assuming that the document set is composed of k clusters, the term-statement matrix A is m × k nonnegative meaning as shown in Equation 1 below so that the objective function represented by Equation 2 below has a minimum value. Decompose into feature matrix W and k × n nonnegative semantic matrix H ^T. In the present embodiment, elements of the i th row and the j th column of the matrix A will be denoted by A _ij .

In Equations 1 and 2, W = [w _ij ], H = [h _ij ], and W = [w ₁ , w ₂ , ..., w _k ].

At this time, in order to update the element values of W and H ^T , the objective function J value according to Equation 2 is smaller than the preset convergence tolerance or exceeds the specified number of repetitions. Repeat.

The preset convergence tolerance can be set to 0.0001 to 0.01, preferably 0.001. The specified number of repetitions can be set to 10 to 100, preferably 50.

A non-negative matrix factorization according to Equation 3 above is performed on the term-sentence matrix A constructed from a given document set to obtain non-negative matrices W and H.

이면 문서 d_i를 군집 x에 할당한다.Next, Equation 4 is used to normalize the matrices W and H, and the cluster label of each data point is determined using the matrix H. E.g,

Then assign document d _i to cluster x.

Next, as a result of applying the non-negative matrix factorization using Equation 3, the A matrix is W and H ^T. Example of decomposition into matrices. In the following example, k = 2, the number of iterations to converge is 50, the convergence tolerance is set to 0.001, W and H ^T The initial values of the matrix were 0.5 respectively.

As a result of performing the non-negative matrix factorization, the j th column vector A _{* j} of the matrix A is a linear combination of the l th column vector W _{* l} of the matrix W and the element H _kj ^T of the matrix H ^T. Is the same as

Next, the non-negative matrix calculation means 320 performs a function of calculating the non-negative semantic feature matrix W and the non-negative semantic variable matrix H.

When the non-negative semantic matrix decomposed from the matrix A by the non-negative matrix factoring means 310 is called W and the non-negative semantic matrix is H, the semantic feature vector W _{* l} of the matrix W The semantic variable H _kj , which is an element of the matrix H, indicates the importance of the semantic feature in the sentence.

Next, the weight recalculation means 330 performs a function of recalculating the weights of terms using the result of the non-negative matrix factorization in order to apply to the document set to be clustered.

Hereinafter, the term weight recalculation of the weight recalculation means 330 will be described in detail.

The term weight recalculation may be performed by Equation 9 and Equation 10 below. [Equation 9] is derived by the following [Equation 6] to [Equation 8].

In Equations 6 to 8, g _a is a weight value of the a-th term, A _ai is the frequency of terms in the a-th term and the i-th document, I _i is the i-th document H _{* i of} H Is a set of k other than ΔH _ki = 0, H _ki ^old is the original value, and H _ki ^new is the value modified by the learning document.

In Equation 9, Δg _a is the amount of change in the average weight of all elements in row a, Δg _a ⁱ is the amount of change in the a-th term weight of the i-th document, and n is the number of entire documents.

In Equation 10, the initial value of g _a ^old is set to 1.

In addition, H _ki ^new can be obtained as shown in Equations 11 to 15 below.

는 정규화된 H_ji ^old이다.In [Equation 11],

Is the normalized H _ji ^old .

로서 k=1,2,...,f^c이다.In Equation 12, c is a serial number of a cluster, and c = 1, 2, ..., e, and e represents the number of clusters. For example, if D _c is the c th cluster, the number of documents included in the c th cluster is | D _c | = f ^c .

K = 1, 2, ..., f ^c .

은 c번째 군집 내에서 의미특징의 중요도의 차이를 나타낸다.In [Equation 13],

Denotes the difference in importance of semantic features within the c th cluster.

In Equation 14, ξ preferably has 0.5 as an adjustment constant.

Equation 16 below is a weight matrix of terms recalculated by Equation 9 above.

The in [Equation 16], the element values of the weighting matrix G is a case that the term of A _{a *} that matches a second term of g _a ^new presence g _a ^new has an element value, otherwise it has a value of 1 .

Finally, the document cluster generation unit 340 performs a function of grouping documents by applying the calculated term weight to the document set.

로 적용한다.The estimated weight for the matrix A is expressed by the following Equation 17 using Equation 16 above.

Apply to.

에 비음수 행렬 인수분해를 이용한 방법 또는 K-평균(K-means) 방법을 이용하여 군집한다.Then, the recalculated weighted matrix

Cluster by using non-negative matrix factorization or K-means method.

In addition, the output unit 400 performs a function of outputting the document cluster generated by the document cluster unit 300 so that a user can check it, which includes a monitor or a printer in a preferred embodiment of the present invention. Although set to, the present invention is not limited thereto.

Hereinafter, a document clustering method using the term weight recalculation according to a preferred embodiment of the present invention will be described with reference to FIG. 2.

2 is a flowchart illustrating a document clustering method using term weight recalculation according to a preferred embodiment of the present invention.

As shown in FIG. 2, a document is first extracted from a set of documents to be clustered by user feedback and a cluster is determined (S10).

In other words, a document is input by the user for recalculation of weights of terms from the document set, and the documents are classified to match the cluster label.

Unlike the feedback related to rewriting terms by extending general queries, the meaning of user feedback in the present embodiment is included in a cluster of results desired by a user in order to recalculate weights for terms so that accurate clustering results can be obtained. It means the feedback to extract the document.

Next, the extracted document set is preprocessed (S20).

The step S20 is performed by removing stop words and extracting roots of the document extracted in the step S10.

Next, the term-phrase matrix A is generated by generating a vector model from the preprocessed document (S30).

After the preprocessing of step S20, the m × n matrix A consisting of a total of m terms and n sentences can be represented by A = [A _{* 1} , A _{* 2} , ..., A _{* n} ], and each row vector A _{* i} is the term-frequency vector of the i-th document.

Next, the term-statement matrix A is decomposed into a non-negative semantic feature matrix and a non-negative semantic variable matrix using non-negative matrix factorization (S40).

In operation S40, the sentence vectors in the extracted document set are represented as a linear sum of a value obtained by multiplying the semantic feature vector by a weighted semantic variable. The semantic feature vector represents the internal feature of the sentence, and the semantic variable represents the importance of the semantic feature in the sentence.

Next, the weight of the term is recalculated by applying the weight recalculation method to the non-negative semantic variable matrix (S50).

The present invention proposes a document clustering method for estimating a weight of a term based on non-negative matrix factorization using a supervised learning method, and clustering documents by applying the estimated weight of the term to a clustered target set. The weight recalculation method according to an example may be applied to all methods of decomposing and clustering original data, such as singular value decomposition or non-negative matrix factorization.

Finally, after applying the calculated term weight to the entire document set, documents are clustered using the document clustering method (S60).

Finally, if the user is satisfied with the clustering result by providing the clustering result of step S60 to the user (S70).

As a result of the determination in step S70, if the user is not satisfied with the clustering result, the procedure proceeds to step S50.

Hereinafter, the experimental results of the document clustering method using the term weight recalculation according to the preferred embodiment of the present invention will be described.

The experiment was performed using data randomly extracted from some of the 20 newsgroups containing 20,000 documents.

The newsgroup consists of 20 different topics, including computer graphics, operating system windows, computer hardware, religion, medicine, and politics, with the same number of articles in each topic.

The evaluation data used in the experiment is shown in the following [Table 1].

Document Set Properties 20 newsgroups Total number of documents 20,000 Documents Used 5,400 Number of clusters 20 Number of clusters used 10 Maximum number of documents in cluster 1,000 Minimum number of documents in cluster 100 Document Count in Intermediate Cluster 500 Document Count in Average Cluster 540

As a measure of performance evaluation, NMI (Normalize Mutual Information) according to Equation 18 was used.

Given two document clusters C and C ', the mutual information MI (C, C') between them is defined as in Equation 18 below.

In Equation 18, p (c _i ) and p (c ' _j ) are probabilities of including documents of a document set in clusters c _i and c' _j , respectively, and p (c _i , c ' _j ) is a document. The probability that the documents in the set are included in clusters c _i and c ' _j at the same time. H (C) and H (C ') are entropy of C and C', respectively.

As shown in Table 2, NMIs of two different clustering methods were compared with increasing the number of clusters from 2 to 10.

k K-means Weighted K-means Nonnegative matrix factorization Weighted Nonnegative Matrix Factorization 2 0.6545 0.6648 0.6874 0.7235 3 0.6898 0.6987 0.6998 0.7432 4 0.6211 0.6541 0.7021 0.7534 5 0.6455 0.6675 0.7212 0.7512 6 0.6635 0.6784 0.6985 0.7224 7 0.6422 0.6544 0.7201 0.7403 8 0.6665 0.6855 0.7032 0.7644 9 0.6633 0.6913 0.6991 0.7411 10 0.6420 0.6702 0.7045 0.7321 Average 0.6543 0.6739 0.7034 0.7413

In Table 2, the K-means method means a standard K-means method, and nonnegative matrix factorization (NMF) is a clustering method proposed by Xu. In addition, the weighted K-means method and the weighted non-negative matrix factorization are methods that are recalculated and applied to the K-means and non-negative matrix factorization methods.

The evaluation was performed using Equation 18 on the results of clustering with and without recalculation of the two clustering methods.

As shown in Table 2, it can be seen that the method of recalculating the weight shows better performance than the method of not calculating the weight. The weighted K-means method has an average NMI of 2.9% higher than the unweighted K-means method, and the weighted non-negative matrix factorization clustering method uses the unweighted non-negative matrix factorization. The average NMI is 5.2% higher than the clustering method.

The K-means method, which does not recalculate weights, exhibits the lowest performance.

The reason why nonnegative matrix factorization clustering method performs better than K-means method is that it is more accurate to group non-negative matrix factorization using nonnegative matrix factorization to reflect the internal structure of data than K-means clustering. Because it can. In addition, the weighted non-negative matrix factorization clustering method reflects the internal structure of the data, and shows the best performance since the weight of the term is recalculated through the learning given by the user to lead to the desired clustering result.

As described above and described with reference to a preferred embodiment for illustrating the technical idea of the present invention, the present invention is not limited to the configuration and operation as shown and described as described above, it is a deviation from the scope of the technical idea It will be understood by those skilled in the art that many modifications and variations can be made to the invention without departing from the scope of the invention. Accordingly, all such suitable changes and modifications and equivalents should be considered to be within the scope of the present invention.

1 is an overall configuration diagram of a document clustering apparatus using term weight recalculation according to a preferred embodiment of the present invention.

2 is an overall flowchart of a document clustering method using term weight recalculation according to a preferred embodiment of the present invention.

Claims (22)

A document clustering device using term weight recalculation, As a preliminary step of document clustering using weighted recalculation of terminology, factoring non-negative matrix by decomposing input document information into individual sentence information, removing stop words, extracting roots, and generating term-sentence matrix A preprocessor 200 for performing a function of grouping documents based on (NMF: Non-Negative Matrix Factorization); And A document cluster unit 300 for factoring the term-phrase matrix into a non-negative matrix, recalculating the weights of terms, and generating a document cluster; Including but not limited to: And the non-negative matrix factorization is performed such that the objective function according to the following equation has a minimum value.

Where J is the objective function and A is the term-statement matrix

W is a non-negative semantic feature matrix (NSFM) and W = [w _ij ], and H ^T is a non-negative semantic variable matrix (NSVM). h _ij ].) The method of claim 1, An input unit 100 receiving document information for recalculating a weight of a term from a user and classifying the received document; And An output unit 400 for outputting the document grouping result generated by the document grouping unit 300 for the user to check; Document clustering apparatus using the term weight recalculation, characterized in that it further comprises. The method of claim 1, The preprocessing unit 200, Sentence decomposition means 210 for decomposing the received document information into respective sentence information; Document cluster setting means (220) for setting the number of document clusters in the received document information; Stopword removing means for removing stopwords from each of the decomposed sentences; A root extracting means 240 for extracting roots in each sentence from which stop words are removed; Term-frequency vector generating means 250 for generating a term-frequency vector according to the frequency of use of the term in each sentence from which stop words are removed; And Term-sentence matrix generating means 260 for generating a term-sentence matrix from the received document information; Document clustering apparatus using the term weight recalculation comprising a. The method of claim 1, The document grouping unit 300, Non-negative matrix factorization means 310 for factoring the term-phrase matrix into a non-negative semantic feature matrix (NSFM) and a non-negative semantic variable matrix (NSVM) ; Non-negative matrix calculation means (320) for calculating the non-negative semantic feature matrix and the non-negative semantic variable matrix; Weight recalculation means (330) for recalculating a weight of terms using the non-negative semantic feature matrix and the non-negative semantic variable matrix; And Document cluster generation means 340 for grouping documents using the recalculated term weights; Document clustering apparatus using the term weight recalculation comprising a. The method of claim 1, The terminology removal is document aggregation apparatus using the term weight recalculation, characterized in that performed through the list of Rijsbergen stopwords. delete The method of claim 1, In order to update the element values of W and H ^T , the term weighting is repeated until the objective function value is smaller than a preset convergence tolerance or exceeds a preset number of repetitions. Document clustering device using recalculation.

The method of claim 7, wherein And the convergence tolerance is 0.0001 to 0.01. The method of claim 7, wherein The document repetition apparatus using the term weight recalculation, characterized in that the number of iterations is 10 to 100. The method of claim 1, As a result of the non-negative matrix factorization, a non-negative semantic feature matrix and a non-negative semantic variable matrix are generated, and the non-negative semantic feature matrix and the non-negative semantic variable matrix generated using the following equation are normalized. Document grouping device using the term weight recalculation.

(In the above equation, w _ij represents an element of the non-negative semantic matrix, and h _ij represents an element of the non-negative semantic matrix.) The method of claim 10,

The document clustering device using term weight recalculation, wherein the document is assigned to cluster x. The method of claim 1, Recalculation of the weight of the term is a document grouping apparatus using the term weight recalculation, characterized in that the following equation.

(A change in the average weight for the entire elements in the equation, Δg _a is a line, Δg _a ⁱ is the change amount of the i-th document in a second term weight, n is the number of the document. Also, g _a is is a weighted value of the a-th term, A _ai is the frequency of the a-th term and the i-th document's term, W and H are the non-negative semantic feature matrix and the non-negative semantic matrix, respectively, generated by factoring the non-negative matrix, I _i is the set of k other than ΔH _ki = 0 in the i-th document H _{* i} of H, H _ki ^old is the original value, and H _ki ^new is the value modified by the learning document.) The method of claim 12, And the initial value of g _a ^old is set to 1. The document grouping apparatus using the term weight recalculation. The method of claim 12, H _ki ^new is a document clustering device using the term weight recalculation, characterized in that calculated by the following equation.

(In the above equation, c is the serial number of the cluster, c = 1, 2, ..., e, e represents the number of clusters,

Is the difference in importance of the semantic features within the c th cluster, and ξ is the control constant.) The method of claim 14, Document grouping device using the term weight recalculation, characterized in that the control constant ξ is 0.5. The method of claim 1, Recalculation of the weight of the term is a document grouping apparatus using the term weight recalculation, characterized in that the following equation.

(Wherein A is a term-sentence matrix,

Is the term-sentence matrix with the recalculated weights applied, and G is the weight matrix of the recalculated terms.) The method of claim 16, remind

A document clustering device using term weight recalculation, characterized in that the generation of the document clustering is performed using a K-means method on a matrix. delete delete delete delete delete

Images