JP2010182253A - Document classification apparatus and document classification method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a document classification support apparatus which prevents useless addition of correct answer and clustering operation. <P>SOLUTION: A document classification apparatus for classifying a plurality of documents on the basis of a given correct category includes: a processor, a memory connected to the processor, an input section, a classification control section, a dispersion degree calculation section, and a degree of progress calculation section. The input section receives the correct category of a document input by a user. The classification control section classifies the plurality of documents into a plurality of groups on the basis of the correct category of the document. The dispersion degree calculation section calculates a dispersion degree shown by the number of documents whose classified groups are changed by the classification. The degree of progress calculation section calculates a degree of progress of the classification. The document classification apparatus displays the result of the classification of the documents by the classification control section, the dispersion degree calculated by the dispersion degree calculation section, and the degree of progress calculated by the degree of progress calculation section. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a document classification device, and more particularly to a document classification device using constrained clustering.

  Conventionally, when a large amount of documents collected by a document retrieval system or the like is classified into several categories according to the contents of each document, a large amount of documents are collected using a mechanical method such as clustering (grouping). An automatic classification method is generally used. Document classification by a mechanical method means that the feature quantity based on the appearance frequency of words existing in a document is an element of the vector of the document, the vector of the document and the vector of other documents This is a method of classifying documents based on similarity or the like.

  A representative clustering method is a K-Means method. The K-Means method is a method of classifying a data set by the following operation when the number of preset clusters (groups) is k (for example, see Non-Patent Document 1). Note that clustering means document grouping or document classification.

  1. From the set of documents, k individuals (documents) are randomly determined and set as the center of gravity of each cluster.

  2. The distance (or similarity) between each individual in the set and the center of gravity is calculated based on the vector of each individual and is classified into the cluster with the nearest center of gravity.

  3. 2. A new centroid in each cluster is determined at random, and the distance (or similarity) between each individual and the new centroid is calculated.

4). 2. 2. the center of gravity used in 3. 1. If the moving distance (for example, the sum of (old centroid-new centroid) ² ) with the new centroid determined in step 2 is not less than a certain threshold value. Return to.

  5. End of the K-Means method.

  Clustering by the K-Means method can classify a large number of documents at a time, but by determining the centroid at random, the classification accuracy is often not sufficient, and the result is not classified as intended by the user. There are many.

  On the other hand, there is a method of improving classification accuracy by incorporating classification intended by the user into the clustering as a constraint. In other words, correctness of an arbitrary document, that is, a classification destination to which an arbitrary document should belong is assigned to the document by the user, and the document to which the correct answer is assigned is used as teacher data to improve the accuracy of clustering. There is a way. This method is called constrained clustering (or semi-supervised clustering) (see Non-Patent Document 2, for example).

  This use of constrained clustering effectively improves the accuracy of clustering. For example, there is a learning type classification method as a clustering method using constrained clustering, that is, a classification method.

  Based on the K-Means method described in Non-Patent Document 1, the learning-type classification method repeats constrained clustering including granting of correct answers by a user to a set of documents, thereby obtaining a clustering result. This is a method that approaches the intended classification.

  In addition to the K-Means method, the learning type classification method includes a learning type classification method based on fuzzy clustering (see, for example, Patent Document 1), or a learning type classification method based on a support vector machine (for example, Patent Document 2). See).

  In the learning type classification method using fuzzy clustering, as in the clustering by the K-Means method, each representative individual is determined in a predetermined number of clusters, and each individual in the entire set is represented as a representative. The affiliation rate is calculated in relation to the individual. When fuzzy clustering is used, each individual has an affiliation rate with respect to each cluster, that is, a numerical value indicating how much belongs. However, when fuzzy clustering is used, it is ambiguous to which cluster each individual belongs to the user, and the classification intended by the user is often not achieved.

  The learning type classification method using support vector machines is a classification method in which cases that belong to one of two classes are regarded as correct cases, and an unknown case belongs to which class is determined based on correct cases It is. The learning type classification method based on the support vector machine requires a work of creating a correct answer case, and this work requires a large cost. In addition, since it is basically a method for identifying two classes, it cannot be identified as a plurality of classes.

Japanese Patent Laid-Open No. 9-305566 JP 2004-021590 A

Trevor Hastie, two others, "The Elements of Statistical Learning: Data Mining, Inference, and Prediction", USA, Springer-Verlag, 2003 Sugar Basu, two others, “Semi-supervised Clustering by Seeding”, Processes of the 19th International Conference on Machine Learning, 2002, pp. 19-26 Christopher D.C. Manning, & Hinrich Schutze, “Fundations of Statistical Natural Language Processing”, USA, MIT Press, 1999

  When the conventional learning type classification method is used, the user cannot predict an appropriate amount of documents to which the user should give a correct answer in order to improve the accuracy of clustering. For this reason, the user may continue uselessly the operation of giving the correct answer to the document.

  For example, a learning-type classification method in which a user assigns correct answers to arbitrary documents and repeats constrained clustering based on the documents given the correct answers (that is, correct answer → constrained clustering → correct answer → constrained clustering) → The operation of sorting documents into the correct classification destination by repeating the above process) continues to give useless correct answers as follows.

  In the first stage of constrained clustering that is repeatedly performed, the accuracy of clustering improves more than the number of documents to which the correct answer is given, as the user gives the correct answer to the document. For example, when five correct answers are given to a set of documents (amount of documents to which correct answers are given: 5), (5 + A) documents are classified as correct by constrained clustering. This (amount of documents to which a correct answer is given + A) indicates the number of documents clustered in the correct classification destination by one constrained clustering, and A is called a learning effect of constrained clustering.

  The learning effect value A of the constrained clustering is a large value in the first stage of starting to give the correct answer. However, when the assignment of correct answers and clustering are repeated and the amount of documents to which a certain correct answer is assigned is reached, the accuracy of constrained clustering does not increase beyond the amount of documents to which the given correct answer is assigned. That is, when the amount of documents to which a certain correct answer is given is reached, the learning effect A becomes a value close to zero.

  The user has no clue to know how much the constrained clustering accuracy is not improved. For this reason, the user is unaware that the learning effect A by the constrained clustering is 0, and repeats the operations of useless correct answer assignment and clustering.

  As described above, when the user does not realize that the learning effect A is 0, the user may give a correct answer to all documents. Originally, the automatic classification method represented by clustering is meaningless because it aims to reduce the work of examining and sorting the contents of all documents.

  It is an object of the present invention to allow a user to recognize the amount of a document that gives a correct answer that improves the accuracy of the most constrained clustering.

  A typical example of the present invention is as follows. That is, a document classification apparatus that includes a processor and a memory connected to the processor and classifies a plurality of documents based on a given correct classification destination, an input unit, a classification control unit, and a divergence calculation And a progress degree calculation unit, wherein the input unit receives the correct classification destination of the document input from the user, and the classification control unit is configured to select the plurality of documents based on the correct classification destination of the document. The documents are classified into a plurality of groups, the divergence degree calculating unit calculates a divergence degree indicated by the number of documents in which the group has changed due to the classification, and the progress degree calculating unit calculates a progress degree of the classification. , The result of classifying the document by the classification control unit, the divergence degree calculated by the divergence degree calculation unit, and the document classification apparatus for displaying the progress degree calculated by the progress degree calculation unit To generate over data.

  According to an embodiment of the present invention, the user does not repeat useless correct answer assignment and clustering when classifying documents.

It is the block diagram which showed the document classification assistance apparatus of embodiment of this invention. It is the flowchart which showed the process of the document classification | category in the information terminal of embodiment of this invention. It is explanatory drawing which showed the clustering result display screen of the document classification assistance apparatus of embodiment of this invention. It is explanatory drawing which showed the table contained in document data DB of embodiment of this invention. It is explanatory drawing which showed the table contained in document index DB of embodiment of this invention. It is the flowchart which showed the process of the clustering of embodiment of this invention. It is explanatory drawing which showed the change of the cluster label of each document before and after clustering of embodiment of this invention. It is explanatory drawing which showed the screen which displays the classification work progress of embodiment of this invention. It is the figure which showed the screen which displays the log | history of a cluster divergence degree by repetition of the clustering of embodiment of this invention. It is explanatory drawing which showed the change of the cluster label of the document of each classification | category by clustering of embodiment of this invention.

  In the embodiment of the present invention, there is a high correlation between the learning effect of constrained clustering by a document given a correct answer by the user and the change of the cluster to which each document belongs in clustering, that is, the change of the cluster label. In the present embodiment, the index when the learning effect by the constrained clustering becomes 0 is indicated by the total value of the number of documents in which the cluster label has changed until the constrained clustering converges.

  The cluster label is an identifier for identifying each classified document group. The document is assigned a cluster label indicating the group of previous documents.

  The constrained clustering method used in the embodiment of the present invention is the K-Means method.

  FIG. 1 is a block diagram showing a document classification support apparatus according to an embodiment of the present invention.

  The document classification support apparatus according to the embodiment of the present invention includes an information terminal 10, two databases, a document data DB 110 and a document index DB 111, and a network 112. The information terminal 10 and the two databases are connected by the network 112, but the information terminal 10 may include two databases.

  The information terminal 10 is a computer including a CPU 101, a memory 102, a keyboard and mouse 103, a display 104, and a data communication unit 109. Further, the information terminal 10 includes a program that provides the functions of the machine classification control unit 105, the cluster divergence calculation unit 106, the progress calculation unit 107, and the document display unit 108.

  The CPU 101 executes document classification processing by executing a program having the functions of the machine classification control unit 105, the cluster divergence calculation unit 106, the progress calculation unit 107, and the document display unit 108. The memory 102 temporarily stores a program executed by the CPU 101 and data necessary for executing the program.

  The keyboard and mouse 103 is a device for a user to input information. The display 104 displays a clustering result, a cluster divergence described later, a classification work progress described later, and the like.

  The machine classification control unit 105 clusters documents input to the information terminal 10. The cluster divergence calculation unit 106 calculates a cluster divergence described later in the clustering process. The progress calculation unit 107 calculates a classification work progress described later in the clustering process. As a result of clustering, the document display unit 108 displays a cluster divergence degree, which will be described later, a classification work progress degree, which will be described later, and the like on the display 104.

  The data communication unit 109 is an interface through which the information terminal 10 performs data communication via the network 112. For example, the data communication unit 109 is a LAN card capable of communication using the TCP / IP protocol. The information terminal 10 communicates with two databases connected to the network 112 via the data communication unit 109.

  The document data DB 110 includes information regarding documents. The document data DB 110 enables searching of the entire document in addition to searching for bibliographic information such as an author or a title.

  The document index DB 111 includes correspondences between documents, keywords, and appearance frequencies. The document index DB 111 provides a keyword list included in a certain document.

  FIG. 2 is a flowchart showing document classification processing in the information terminal according to the embodiment of the present invention.

  An outline of the document classification process executed by the machine classification control unit 105, the cluster divergence calculation unit 106, the progress calculation unit 107, and the document display unit 108 will be described with reference to FIG.

  First, the user inputs a set of documents previously collected by a document search system or the like to the document classification support apparatus using the keyboard and mouse 103. At this time, the user may input a clustering condition such as the number of clusters in addition to the set of documents. In addition to the number of clusters, the clustering conditions include, for example, representative keywords that describe each cluster. Note that the center of gravity may be determined based on the input keyword. The set of input documents is stored in the document data DB 110 and the document index DB 111.

  The machine classification control unit 105 acquires a keyword and an appearance frequency of each document in the document set from the document index DB 111 (S201).

  Next, the machine classification control unit 105 performs clustering according to the appearance frequency of the keyword of each document acquired in S201. Subsequently, the cluster divergence calculator 106 calculates the cluster divergence (S202). A method of calculating the cluster divergence will be described later with reference to FIGS.

  Next, the progress degree calculation unit 107 calculates the classification work progress degree from the cluster divergence degree obtained in S202 and the number of correct answers given to the document. A method of calculating the classification work progress will be described later with reference to FIG. Subsequently, the document display unit 108 displays the cluster divergence and the classification work progress calculated in S202 on the display 104 as a result of the clustering in S202 (S203).

  A specific example of the screen displayed on the display 104 will be described later with reference to FIG. Note that the degree of progress of the classification work in S203 is calculated by setting the number of correct answers to 0, since the correct answer has not yet been given immediately after the first set of documents is input.

  Based on the cluster divergence and the classification work progress, which are the clustering results displayed on the display 104 in S203, the user determines whether or not the document classification work can be completed (S204). It should be noted that the user determines whether or not to end depending on whether or not the cluster divergence has a sufficiently low value or whether or not the classification work progress has a sufficiently high value.

  When the result of clustering determined in S204 is not the result desired by the user, and the user inputs to the information terminal 10 that the classification operation cannot be completed (when determined No in S204), The user gives a correct answer to the documents in each cluster (S205). After S205, the document classification process returns to S202.

  In addition, for the operation to give a correct answer to a document, a “fix” operation that clearly indicates that the document is correctly classified into a correctly classified document, and a “move” that specifies a correct cluster for the incorrectly classified document There is an operation.

  When the result of clustering determined in S204 is a result desired by the user and the user inputs to the information terminal 10 that the classification work can be completed (when determined Yes in S204), The document classification work using the document classification support apparatus of the present invention is terminated.

  Hereinafter, the display screen of the clustering result in S203, the details of the document data DB 110 and the document index DB 111, and the details of the calculations associated with S202 and S203 of FIG. 2 will be described.

  FIG. 3 is an explanatory diagram illustrating a clustering result display screen 301 of the document classification support apparatus according to the embodiment of this invention.

  The clustering result display screen 301 includes an area for displaying the clustering result and an area for displaying the classification work progress and the cluster divergence. Further, a clustering button 302 for clustering (S202 in FIG. 2), a fixed button 303 for clearly indicating that the document is correctly classified into a correctly classified document, and a correct cluster for an incorrectly classified document. A move button 304 for designating is included in the clustering result display screen 301.

  The area for displaying the clustering result includes a clustering result table 305. The clustering result table 305 displays document classification results by two axes, a vertical axis and a horizontal axis. The vertical axis and the horizontal axis are the results of classifying a set of documents from different viewpoints. For example, the horizontal axis may be the result of classification by clustering, and the vertical axis may be the result of classification by document bibliographic information stored in the document data DB 110. Further, both the vertical axis and the horizontal axis may be the result of classification by clustering. Based on the clustering result table 305, the user recognizes to which classification each document belongs as a result of clustering.

  The vertical classification 306 and the horizontal classification 307 indicate the names of the classifications on the vertical axis and the horizontal axis, respectively. In the document list 308, names of documents belonging to the vertical classification 306 and the horizontal classification 307 are displayed as a list.

  The fixed check box 309 selects a document that the user wants to “fix” when the affiliation classification of the document corresponding to the fixed check box 309 is correct. Then, when the user operates the fixed button 303, a check is input to the fixed check box 309 of the selected document.

  If the affiliation classification of the document corresponding to the fixed check box 309 is incorrect, the user selects the document to be “moved” and operates the “move” button 304 to perform the “move” operation. The user designates a correct classification destination by operating the move button 304, the selected document is moved to the designated classification destination, and a check is automatically input to the fixed check box 309 of the selected document.

  In the “move” operation, after the user operates the move button 304, a screen for designating a new classification destination of the selected document is displayed, and the user inputs the classification destination on the displayed screen. The selected document may be moved to a new classification destination, and a check may be automatically input to the fixed check box 309. Further, after the user visually moves the selected document display to a new classification destination by drag and drop and operates the move button 304, a check is input to the fixed check box 309 of the selected document. Also good.

  The process of S205 shown in FIG. 2 is executed when the user performs a fixing and moving operation on each document using the fixed button 303 and the moving button 304.

  The progress / cluster divergence display area includes a progress display 310 and a cluster divergence history display 311.

  The progress display unit 310 displays the classification work progress calculated by the progress calculation unit 107. The user can recognize the progress of clustering for the document by the display of the progress display unit 310.

  The cluster divergence history display unit 311 displays the cluster divergence calculated by the cluster divergence calculation unit 106 and obtained by each clustering process at the time of display, that is, the total value of the number of documents whose cluster labels have changed. The user can recognize the number of documents whose cluster labels have changed in each clustering by the display of the cluster divergence history display unit 311.

  A method of calculating the classification work progress will be described later with reference to FIG. 8, and a method of calculating the cluster divergence will be described later with reference to FIGS.

  Next, database data used in the processing shown in FIG. 2 will be described.

  FIG. 4 is an explanatory diagram illustrating a table included in the document data DB 110 according to the embodiment of this invention.

  A table storing document data in the document data DB 110 includes a document number 401, an author 402, an issue year 403, a classification 404, and a full text 405.

  A document number 401 is an identifier of a stored document. The author 402 is the name of the document author. The issue year 403 is the year that the document was issued. A classification 404 is a classification given to the document. The full text 405 is the content of the full text of the document. Note that the column elements of the table shown in FIG. 4 may be added or changed depending on the type of the target document.

  FIG. 5 is an explanatory diagram illustrating a table included in the document index DB 111 according to the embodiment of this invention.

  The document index DB 111 includes an index 501 for clustering documents according to keywords in the document.

  The index 501 is used to calculate the similarity between a document and a cluster.

  The index 501 includes a document number 502 and a list 503. A document number 502 is an identifier of a stored document. The same value as the document number 401 in the document data DB 110 is shown.

  The list 503 includes a (keyword number, frequency) pair. The keyword number is an identifier of a keyword included in the corresponding document. The frequency is the number of times that the keyword appears in the document (appearance frequency).

  The similarity can be calculated based on the keyword and the appearance frequency (see, for example, “Information Retrieval Algorithm” by Kenji Kitaken et al., Kyoritsu Shuppan, 2002). For example, a high degree of similarity is calculated between documents having a similar appearance frequency of a certain keyword.

  The details of the cluster divergence calculation method in S202 shown in FIG. 2 and the classification work progress calculation method in S203 will be described below.

  First, constrained clustering based on the K-Means method is used for clustering in S202 shown in FIG. The cluster divergence is calculated by the cluster divergence calculator 106 simultaneously with the constrained clustering. Details of the processing in S202 are shown in FIG.

  FIG. 6 is a flowchart illustrating clustering processing according to the embodiment of this invention.

  After a set of documents is input to the information terminal 10 (S201), or a correct answer is given to the document using the fixed button 302 and the move button 304 shown in FIG. 3 (S205), the user displays the clustering result display screen 301. When the clustering button 302 is operated at S202, S202 is started. In the process of S202, the machine classification control unit 105 executes constrained clustering based on the document set and the data included in the document index DB 111.

  The cluster divergence calculation unit 106 sets the value of the cluster divergence to 0 before the restricted clustering in S202 is executed (S601). The cluster divergence calculation unit 106 stores the cluster label of each document (S602). The cluster labels of this embodiment correspond to the classification names of the vertical classification 306 and the horizontal classification 307 in the clustering result table 305.

  After the process of S602, the machine classification control unit 105 executes clustering (S603). For clustering, constrained clustering based on the K-Means method is used. The constrained clustering algorithm is shown below.

  1. A cluster having a document with a check input in the fixed check box 309 shown in FIG. 3 calculates a centroid using all the documents with the check input in the cluster. For a cluster with no document for which a check is input, the center of gravity is calculated using all the documents belonging to the cluster.

  2. The degree of similarity between each document and the centroid is calculated, and each document is classified into a cluster of centroids indicating the highest similarity, that is, the closest. However, the document whose check is input to the fixed check box 309 is not classified according to the similarity, and the cluster label is maintained as it is.

  3.2. A new center of gravity is calculated among the documents belonging to the cluster classified in (1).

  Above 1. ~ 3. When the constrained clustering is completed, the cluster divergence calculating unit 106 compares the cluster label of each document stored in S602 with the new cluster label of each document obtained in S603, and the number of documents whose cluster label has changed. And the calculated number of documents is added to the cluster divergence. Further, the cluster divergence calculation unit 106 also calculates the number of documents in which the cluster label has changed for each cluster, and adds the number of documents calculated for each cluster to the cluster divergence for each cluster (S604).

  FIG. 7 is an explanatory diagram showing changes in the cluster label of each document before and after clustering according to the present embodiment.

  FIG. 7 shows the results when 20 documents are clustered with 3 clusters.

  The left table 701 is a list of cluster labels of each document before clustering (that is, the cluster label stored in S602), and the right table 702 is a list of cluster labels after the clustering result (that is, S604). Equivalent to the cluster label obtained in step 1).

  When the left table 701 and the right table 702 are compared, the cluster divergence calculation is performed because the cluster labels of ten documents indicating the document numbers 4, 5, 6, 9, 10, 11, 12, 13, 19, and 20 have changed. The unit 106 adds 10 to the cluster divergence. Also, two documents (document numbers 19 and 20) in cluster 1, three documents (document numbers 4, 5, and 6) in cluster 3, and five documents (document numbers 9, 10, 11, 12, and 13) in cluster 3 Therefore, the cluster divergence calculation unit 106 adds to each cluster divergence of each cluster.

  After S604, the machine classification control unit 105 determines whether the clustering has converged (S605). As the convergence determination method of the present embodiment, for example, the convergence determination method in the K-Means method disclosed in Non-Patent Document 1 is used. That is, in the convergence determination method of the present embodiment, the administrator predetermines a threshold value of the change amount of the center of gravity obtained by each clustering, and based on the change amount of the center of gravity obtained by each clustering and the determined threshold value. This is a method for determining whether or not the convergence has occurred.

  When it is determined in S605 that the clustering has not converged (No is determined in S605), the same processing from S602 is repeated. If it is determined in S605 that the clustering has converged (if determined as Yes in S605), the clustering process illustrated in FIG. 6 ends.

  The clustering in S202 is repeated until it is determined in S605 that it has converged (this repetition is referred to as a loop). For example, when a loop occurs five times and the cluster divergence in each loop is 10, 9, 6, 4, 0, the final cluster divergence is (10 + 9 + 6 + 4 + 0) = 29. The calculation method of the cluster divergence for each cluster is the same calculation method.

  After the clustering process in S202 is completed, the document display unit 108 displays the cluster divergence and the classification work progress calculated by a method described later (S203).

  2 and 6, the user can recognize the cluster divergence and the classification work progress for each clustering.

  The classification work progress is calculated by the progress calculation unit 107 using Equation 1.

  The cluster divergence shown in Equation 1 is the cluster divergence obtained in S202. The number of loops shown in Equation 1 is the number of repetitions of S602 to S605 shown in FIG. The number of fixed documents shown in Equation 1 is the number of documents whose check is input to the fixed check box 309 among the documents displayed in the clustering result table 305. The total number of documents shown in Equation 1 is the number of documents included in the entire document.

  Α shown in Equation 1 is a weighting coefficient indicating which of the clustering and the operation for giving the correct answer of the user is emphasized, and is set in advance by the administrator of the document classification support apparatus in the embodiment of the present invention. Alternatively, the user may specify. As for α, it is desirable that the set value range is 0 ≦ α ≦ 1.0.

  The classification work progress is determined based on two indicators. One index is how many percent of the set of documents to be classified are given correct answers (correct answer progress), and the other index is the classification intended by the user by clustering. (Clustering stability). In the formula for calculating the classification work progress in Equation 1, the term on the left side of the numerator corresponds to “How many percent of the set of documents has been given the correct answer”, and the term on the right side is the classification intended by the user. Is equivalent to?

  The value of α is determined by which one of the two indexes, the progress degree of correct answer assignment and the clustering stability, is important by the manager. The administrator increases the value of α when it is desired to make the correct assignment progress important, and the administrator decreases the value of α when the clustering stability is important.

  When the classification work progress degree obtained by the calculation formula shown in Equation 1 is a large value, the user determines that the classification work is progressing, and when the classification work is small, the user determines that the classification work is not progressing.

  As shown in the classification work progress display unit 310 shown in FIG. 3, the classification work progress display method displays one result according to the calculation formula shown in Equation 1, The two terms may be displayed separately. That is, the classification work progress display unit 310 may be displayed separately on the screen shown in FIG.

  FIG. 8 is an explanatory diagram showing a screen displaying the classification work progress of the embodiment of the present invention.

  The screen shown in FIG. 8 displays the correct answer assignment progress level and the clustering stability level. When the screen shown in FIG. 8 is used, it indicates that the classification work is progressing as the numerical value of the correct assignment progress is large, and the classification work is advanced as the numerical value of the clustering stability is small.

  FIG. 9 is an explanatory diagram showing a screen that displays a history of cluster divergence by repeating clustering according to the embodiment of this invention.

  In this embodiment, the history of cluster divergence is represented by a bar graph.

  The horizontal axis of the explanatory diagram shown in FIG. 9 indicates the value of the cluster divergence obtained by one clustering process (corresponding to one process of S202). 1st, 2nd,. . . The n-th display gives the correct answer to the document and indicates the result obtained in the repeated operation of clustering (S202 to S205).

  The vertical axis of the explanatory diagram shown in FIG. 9 is the value of the cluster divergence obtained in the corresponding clustering. The explanatory diagram shown in FIG. 9 is a case where a set of documents is classified into three clusters, and the bar graph can identify the breakdown of cluster divergence corresponding to each classification (corresponding to each cluster in clustering) by color. To display.

  With the display shown in FIG. 9, the user confirms that the bar graph decreases as the clustering process is repeated (from the left to the right on the horizontal axis), that is, the cluster divergence approaches 0, and the classification of the document set is It is possible to recognize that the user is approaching the intended classification.

  In addition, the user can confirm which class has not converged from the cluster divergence of each class, that is, whether it is insufficient to give a correct answer to the document. It is possible to make a policy on whether to give the correct answer. Note that classification with insufficient correct answers is unlikely to reduce the cluster divergence even if clustering is repeated.

  FIG. 10 is an explanatory diagram showing changes in cluster labels of documents of each classification by clustering according to the embodiment of this invention.

  FIG. 10 is a display of the clustering result table 305 different from the clustering result table 305 shown in FIG. Based on the clustering result table 305 illustrated in FIG. 10, the user recognizes a classification in which documents belonging to the vertical classification and the horizontal classification have changed due to clustering.

  The display of values in the clustering result table 305 shown in FIG. 10 is indicated by a value 1001 and a background 1002.

A value 1001 indicates the number of documents belonging to the vertical classification and horizontal classification in the clustering result table 305. A background 1002 is changed by changing the background color of the classification in which the number of documents has changed before and after clustering. Emphasize the classification. The emphasized classification is a classification in which a document that was not present before clustering is newly added to the classification. Therefore, the classification in which the number of documents belonging to the document is reduced due to the movement of the document to another classification by clustering is not emphasized. In addition, the classification in which the number of belonging documents is reduced and a new document is added is emphasized as compared to before clustering.

  Note that the change of the classification to which the document belongs is determined by comparing the classification before clustering stored in the process of S602 shown in FIG. 6 with the classification after clustering in the process of S604.

  Note that the clustering result table 305 shown in FIG. 10 displays the number of documents to which the user belongs, and emphasizes the background 1002, but displays the list of documents in the clustering result table 305 shown in FIG. 3 and emphasizes the background. May be.

  As described above, according to the embodiment of the present invention, by displaying the classification work progress, the user can consider both the number of correct answers given to the document and the clustering stability based on the cluster divergence. The degree of work progress can be recognized, and it is possible to eliminate waste of repeating correct operation and clustering operation that are not effective for clustering.

  Further, by displaying the change in classification due to clustering for each classification, it becomes easier to make a policy as to which classification should be given the correct answer, and effective clustering becomes possible.

10 Information terminal 101 CPU
102 Memory 103 Keyboard / Mouse 104 Display 105 Machine Classification Control Unit 106 Cluster Divergence Calculation Unit 107 Progress Level Calculation Unit 108 Document Display Unit 109 Data Communication Unit 110 Document Data DB
111 Document Index DB
112 network

Claims (10)

A document classification device comprising a processor and a memory connected to the processor, and classifying a plurality of documents based on a given correct classification destination,
An input unit, a classification control unit, a divergence calculation unit, and a progress calculation unit,
The input unit accepts the correct classification destination of the document input from the user;
The classification control unit classifies the plurality of documents into a plurality of groups based on a correct classification destination of the document,
The divergence calculator calculates a divergence indicated by the number of documents in which the group has changed due to the classification,
The progress calculation unit calculates the progress of the classification,
Generating data for displaying a result of classifying the document by the classification control unit, a divergence degree calculated by the divergence degree calculation unit, and a progress degree calculated by the progress degree calculation unit; Document classification device. The divergence calculation unit calculates the divergence for each classification for each group,
The document classification apparatus according to claim 1, wherein data for displaying the divergence calculated in the plurality of classifications on a single screen is generated.   The progress degree calculation unit calculates the progress degree based on the calculated divergence degree, the number of times of classification, and the number of documents in which the correct classification destination is input. Document document classification device.   2. The document classification apparatus according to claim 1, wherein data for displaying the classification result, the calculated degree of divergence, and the calculated degree of progress for each classification is generated. .   2. The document according to claim 1, wherein data for displaying the group in which the number of documents belonging to the classification has changed and the group in which the number of documents belonging to the group has not changed is generated in an identifiable manner. Classification device. A document classification method in a document classification device that includes a processor and a memory connected to the processor and classifies a plurality of documents based on a given correct classification destination,
The document classification method is:
The processor accepts the correct classification destination of a document input by a user;
The processor classifies the plurality of documents into a plurality of groups based on a correct classification destination of the document;
Storing the classification result in the memory;
The processor calculates a divergence indicated by the number of documents in which the group has changed due to the classification;
The processor calculates the progress of the classification;
The processor stores, in the memory, the classified result, the calculated divergence, and the calculated progress.
The document classification method, wherein the processor generates data for displaying the classified result, the calculated degree of divergence, and the calculated degree of progress. The processor calculates the divergence per classification for each group,
The document classification method according to claim 6, wherein the processor generates data for displaying the divergence calculated in the plurality of classifications on a single screen.   7. The document according to claim 6, wherein the processor calculates the degree of progress based on the calculated divergence, the number of times of classification, and the number of documents to which the correct classification destination is input. Classification method.   The said processor produces | generates the data for displaying the classified result, the calculated divergence degree, and the calculated progress degree for every classification | category. Document classification method.   7. The processor generates data for distinguishably displaying a group in which the number of documents belonging to the group has changed due to the classification and a group in which the number of documents belonging to the group has not changed. Document classification method described in 1.