JP2005266866A - Document classifying device and classification system generating device and method for document classifying device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To automatically construct the classification system of a hierarchical structure from a flat classification frame. <P>SOLUTION: A sample document inputting part 141 receives a sample document associated with a classification frame from a sample document storing part 13, and supplies it to an initial classification system generating part 142 (S10). An initial classification system generating part 142 refers to the sample document for clustering classification frames, and generates an initial classification system by defining the cluster as an upper rank classification frame and the classification frame to which the cluster is belonging as a terminal classification frame, and stores it in a classification system storing part 143 (S11). An evaluating part 144 automatically classifies the sample document by referring to the temporary classification system stored in the classification system storing part 143 (S12), and evaluates the classification result (S13). When there is any upper rank classification frame whose precision is lower than a reference value, a classification system changing part 145 changes the classification system. The cluster corresponding to the upper rank classification frame whose precision is lower than the reference value and a cluster similar to the cluster are united into one cluster for re-clustering, and the cluster is generated, and the classification system is changed (S15). <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a hierarchization technique of a classification system for classifying documents, and easily classifies a classification system or optimizes a hierarchy.

  A new arrival or attention document is automatically classified according to a predetermined classification system. For example, a classification frame (also called a category) is set for each specific concept or topic, and a document is classified into a classification frame using a word vector or a keyword. The user can concentrate his effort on a desired classification frame (also called a category), which is very convenient.

  By the way, if the classification system referred to at the time of classification has a hierarchical structure, the listability of the results is enhanced and the classification error is strong. The reason for the latter is that even if there is a classification error, it is highly likely that the document being searched for can be found relatively easily by referring to another classification frame in the same hierarchy. In general, when performing document classification mechanically, natural language itself or classification definition is ambiguous, so it is difficult to achieve high accuracy, and it is strongly desired to be resistant to classification errors.

  However, it is expensive for the user to construct a hierarchical classification system. With a non-hierarchical so-called flat classification system, the user only needs to define a classification frame. On the other hand, in order to construct a hierarchical structure, it is necessary for the user to determine the vertical relationship between the classification frames, the similarity relationship, and the like for each classification frame. It is difficult for a user to judge such a relationship with an actual document or a set of documents because the criteria are not clear.

Patent Document 1 discloses a method of referring to or reusing category structure information generated in the past for the purpose of solving this problem. However, with this method, information suitable for reuse must already be accumulated, and cannot be applied when there is no such information to be reused. Moreover, even if the information is appropriate in the past, it cannot always be said that it is appropriate when applied this time, and a timely category structure cannot be constructed.
JP 2003-216622 A

  The present invention has been made in consideration of the above circumstances, and an object thereof is to provide a hierarchization technique for a classification system that can easily construct a hierarchical structure of classification even from a sample document prepared by a user.

  The essence of the present invention is that the system (device or device group) automatically constructs a hierarchical classification system based on a set of flat classification frames defined by the user. In the configuration example of the present invention, the following processing is performed.

[Step S1]:
The user defines a classification frame with a set of sample documents to be classified. At this time, it is not necessary for the user to prepare information regarding the structure.

[Step S2]:
The system divides a given classification frame into, for example, 5 to 20 clusters using an existing clustering technique or the like. In the case of document clustering, the target is generally in a document, but the target here is not a sample document but a classification frame.

[Step S3]:
Using the sample document as an input, the cluster prepared in step 2 is automatically classified according to the classification system.

  When the system performs automatic classification, first, the degree of relevance with each cluster included in the highest hierarchy is calculated for the input document and distributed to the cluster with the highest degree of relevance. Next, if the cluster has a lower hierarchy, the same thing is done for that hierarchy. These processes are continued until they are assigned to the classification frame defined by the user.

  The degree of association between the input document and the cluster is defined by the degree of association between the word vector of the document and the word vector of the cluster. The size of the word vector is obtained from a statistic regarding the appearance frequency of words in the cluster. At this time, all sample documents included in the hierarchy to which a certain cluster belongs are set as a population parameter.

  Since the sample documents are different for each hierarchy, the word vectors have different sizes. Therefore, when the relevance level of a document is obtained from the highest level, it can be expected that an appropriate relevance level is obtained for each level. That is, the classification result is not determined simply by the presence or absence of the keyword or the word score that is uniformly determined, but is classified according to the hierarchical structure of the classification system.

[Step S4]:
Evaluate the accuracy of automatic classification results. A case where the sample document is classified into a cluster including a predetermined classification frame is regarded as a correct answer, and an F value that is a harmonic average of the recall rate or the recall rate and the matching rate is measured.

[Step S5]:
When the accuracy of the classification result is lower than the reference value, attention is paid to a cluster whose classification accuracy is lower than the reference value and a cluster similar thereto. These clusters are put together to set a virtual hierarchical classification frame, and the classification frame is arranged below this virtual classification frame. Clustering is again performed on these classification frames, and the process returns to step 3. A predetermined reference value is used. It may be determined based on the number of classification elements and various conditions, or the user may be able to change the setting.

[Step S6]:
When sufficiently good accuracy is obtained, each cluster is set as a virtual hierarchical classification frame, and the classification frame is arranged below it.

  As a result, a classification system having a hierarchical structure is constructed. The classification frame defined by the user becomes the classification frame at the end of the hierarchical structure, and the other classification frames are set by the system. The hierarchical structure is set so that a classification error approaches a minimum when automatic classification is performed using a sample document as an input document.

  Note that the hierarchical structure of the classification system may be prepared in advance, and the modified structure may be optimized using steps S3 to S6. That is, the user designates a classification system having a hierarchical structure as an input, and as a result, a more accurate classification system is obtained.

  The present invention will be further described.

  According to one aspect of the present invention, in order to achieve the above-described object, a sample document for each non-hierarchical terminal classification frame is input to a classification system generation apparatus that generates a hierarchical classification system of a document classification apparatus. Clustering the non-hierarchical terminal classification frame with reference to the sample document, and placing the cluster obtained as a result of the clustering as a high-level classification frame above the non-hierarchical terminal classification frame, an initial hierarchy Initial hierarchical structure classification system generation means for generating as a structural classification system; hierarchical structure classification system storage means for storing an initial hierarchical structure classification system or a modified hierarchical structure classification system; and stored in the hierarchical structure classification system storage means Classification accuracy evaluation means for classifying the sample document with reference to a hierarchical classification system, and determining a higher classification frame whose classification accuracy is lower than a reference value; When a higher classification frame lower than the reference value is determined, the cluster of the higher classification frame evaluated by the classification accuracy evaluation means as having an accuracy lower than the reference value is integrated with other clusters and reorganized into one cluster. The terminal classification frames included in the integrated one cluster are reclustered with reference to the sample document included in the terminal classification frame, and the cluster obtained as a result of the reclustering corresponds to the one cluster. Hierarchical structure changing means for changing the hierarchical structure classification system stored in the hierarchical structure classification system storage means so as to be an upper classification frame lower than the upper classification frame is provided; when a predetermined end condition is not satisfied Repeats the determination by the classification result evaluation means and the change of the hierarchical structure classification system by the hierarchical structure changing means, and a predetermined end condition is satisfied. The hierarchy classification system stored in the hierarchy classification system storage means can have used as the classification system for the document classification apparatus.

  In this configuration, clusters are generated by clustering starting from a non-hierarchical, that is, flat classification frame, a hierarchical structure classification system is prepared using these clusters as upper classification frames, and the classification accuracy is higher than the reference value. Focusing on the classification frame, that is, the cluster, it is integrated with other clusters and then reclustered to expand the hierarchy. By extending the hierarchy, it is expected that the upper classification frame whose accuracy is lower than the reference value will be improved. By simply performing classification using a provisional hierarchical structure classification frame, a hierarchical structure classification frame with improved classification accuracy can be easily generated. Note that a predetermined reference value is used. It may be determined based on the number of classification elements and various conditions, or the user may be able to change the setting.

  In this configuration, the other cluster integrated with the cluster whose accuracy is lower than the reference value can be a cluster similar to the cluster whose accuracy is lower than the reference value.

  Some clusters include only a single classification frame.

  Further, the termination condition may be that none of the clusters is evaluated as having a classification accuracy lower than the reference value. The hierarchy conditions, the number of times of re-clustering, the number of times of classification, etc. in the current hierarchical structure classification system may be used as the termination conditions.

  In addition, the present invention can be realized not only as a classification system generation apparatus that generates a hierarchical structure classification system of a document classification apparatus, but also as a document classification apparatus including such a mechanism. Further, it can be configured as a document classification device including a mechanism for optimizing the existing hierarchical structure classification system of the document classification device.

  The present invention can be realized not only as an apparatus or a system but also as a method. Of course, a part of the invention can be configured as software. Of course, software products used to cause a computer to execute such software are also included in the technical scope of the present invention.

  These and other aspects of the invention are set forth in the appended claims and will be described in detail below with reference to examples.

  According to the present invention, it is possible to realize a hierarchical classification that is expected to have a higher effect with the same effort as a flat classification for the user. In addition, it is not necessary to construct hierarchical information in advance, and a hierarchical classification system that optimally represents a given classification frame can be automatically obtained each time a classification system is constructed.

  Examples of the present invention will be described below.

  FIG. 1 shows a document classification device 1 to which the present invention is applied, FIG. 2 shows a classification system hierarchization unit 14 which is a main part of the document classification device 1, and FIG. Shows the operation. In this example, the classification system hierarchization unit 14 is configured as one mechanism of the document classification device 1, but may be configured as a single device. For example, a hierarchical document classification system may be generated as a single device and supplied to the document classification device 1 via a recording medium or a communication medium.

  In FIG. 1, the document classification device 1 includes a document input unit 10, a document classification unit 11, a classified document storage unit 12, a sample document storage unit 13, and a classification system hierarchization unit 14. The document input unit 10 inputs a group of documents prepared in advance, a group of documents input periodically, and a document input sequentially, and the documents are transmitted via a recording medium or a communication medium. . The document may be generated from the site of the document classification device 1. The document classification unit 11 classifies documents with reference to a hierarchical structure classification system, for example, a classification system as shown in FIG. The hierarchical structure classification system is composed of a terminal classification frame and an upper classification frame, and has a tree structure. For classification of documents, keywords or word vectors can be used. In this example, a word vector is used and a word vector is assigned to each classification frame (upper classification frame and terminal classification frame), but the present invention is not limited to this. The document starts from the root classification frame, is matched with the word vector of each branch (branch destination classification frame), is classified into the most appropriate branch, and is also matched with the branch word vector in the next hierarchy, It is classified into an appropriate branch and finally classified into one of the terminal classification frames. The classified documents are stored in the classified document storage unit 12. The classified document storage unit 12 may store the document body, or may store document attributes, classification frames, document body addresses, and the like, and the document body may be stored in another storage unit.

  The classified documents can be browsed using a predetermined browsing mechanism. You can specify a classification frame at an arbitrary level so that you can browse the attributes, summaries, etc. of the documents in the classification frame. Also good. Various other mechanisms of the classification device can be employed.

  The classification system hierarchization unit 14 is for generating a hierarchical structure classification system used in the document classification unit 11. The classification system hierarchization unit 14 generates a hierarchical structure classification system using the sample documents stored in the sample document storage unit 13. The sample documents stored in the sample document storage unit 13 are divided in advance into a plurality of classification frames (terminal classification frames) arranged flat by a user or the like. Details of the classification system hierarchization unit 14 will be described with reference to FIGS.

  FIG. 2 shows a configuration example of the classification system hierarchization unit 14. In FIG. 2, the classification system hierarchization unit 14 includes a sample document input unit 141, an initial classification system generation unit 142, a classification system storage unit 143, an evaluation unit 144, a classification system change unit 145, and the like. These units are actually executed by a program process. The sample document input unit 141 corresponds to step S10 in FIG. 3, and the initial classification system generation unit 142 similarly corresponds to step S11. 144 corresponds to steps S12 to S14, and the classification system changing unit 145 also corresponds to step S15.

  The classification system hierarchization unit 14 will be further described with reference to FIGS. 2 and 3. 2 and 3, the sample document input unit 141 receives the sample document related to the classification frame from the sample document storage unit 13 and supplies it to the initial classification system generation unit 142 (S10). The initial classification system generation unit 142 clusters the classification frames with reference to the sample document, generates an initial classification system using the cluster as the upper classification frame and the classification frame to which the cluster belongs as the terminal classification frame, and stores the initial classification system in the classification system storage unit 143. (S11). The evaluation unit 144 automatically classifies the sample document with reference to the provisional classification system stored in the classification system storage unit 143 (S12), and then evaluates the classification result (S13). If the accuracy of any of the upper classification frames is not lower than the reference value, the provisional classification system is determined as the final classification system (S14, S16). If there is a higher classification frame whose accuracy is lower than the reference value, the classification system changing unit 145 changes the classification system. A cluster corresponding to a higher classification frame whose accuracy is lower than the reference value and a cluster similar to this cluster or a terminal classification frame (a cluster including only one terminal classification frame) are integrated into one cluster, and the integrated cluster is the same as in step S11. Re-clustering is performed in the range of (1) to generate clusters and change the classification system (S15). That is, an upper classification frame corresponding to a cluster generated by reclustering is prepared below the upper classification frame corresponding to the integrated cluster, and the terminal classification frame is arranged below the upper classification frame. As a result, the depth of the hierarchy increases.

  S13 to S15 are repeated until there is no higher classification whose accuracy is lower than the reference value, and when the accuracy of all the higher classifications finally becomes good, the classification system is determined (S16). It should be noted that other conditions may be re-examined instead of the termination condition that the accuracy of all the upper classifications becomes good. For example, it may be adopted that the depth of the classification system has reached a predetermined value, that the number of repetitions of steps S13 to S15 has reached a predetermined value, that the processing time has reached a predetermined value, etc. good.

  Each of the above steps will be described in more detail.

[Step S10]: The user defines a classification frame by a set of sample documents to be classified. At this time, it is not necessary for the user to prepare information regarding the structure.

[Step S11]: The system divides the given classification frame into, for example, 5 to 20 clusters using an existing clustering technique or the like. In the case of document clustering, the target is generally in a document, but the target here is not a sample document but a classification frame.

[Step S12]: The sample document is input, and the cluster of step S11 is automatically classified according to the classification system.

  When the system performs automatic classification, first, the degree of relevance with each cluster included in the highest hierarchy is calculated for the input document and distributed to the cluster with the highest degree of relevance. Next, if the cluster has a lower hierarchy, the same thing is done for that hierarchy. These processes are continued until they are assigned to the classification frame defined by the user.

  The degree of association between the input document and the cluster is defined by the degree of association between the word vector of the document and the word vector of the cluster. The size of the word vector is obtained from a statistic regarding the appearance frequency of words in the cluster. At this time, all sample documents included in the hierarchy to which a certain cluster belongs are set as a population parameter.

  Since the sample documents are different for each hierarchy, the word vectors have different sizes. Therefore, when the relevance level of a document is obtained from the highest level, it can be expected that an appropriate relevance level is obtained for each level. That is, the classification result is not determined simply by the presence or absence of the keyword or the word score that is uniformly determined, but is classified according to the hierarchical structure of the classification system.

[Step S13]: The accuracy of the automatic classification result is evaluated. A case where the sample document is classified into a cluster including a predetermined classification frame is regarded as a correct answer, and an F value that is a harmonic average of the recall rate or the recall rate and the matching rate is measured.

[Steps S14 and S15]: When the accuracy of the result is lower than the reference value, attention is paid to a cluster whose classification accuracy is lower than the reference value and a cluster similar thereto. These clusters are put together to set a virtual hierarchical classification frame, and the classification frame is arranged below this virtual classification frame. Clustering is again performed on these classification frames, and the process returns to step S12.

[Steps S14 and S16]: When sufficiently high accuracy is obtained, each cluster is set as a virtual hierarchical classification frame, and the classification frame is arranged below it.

  As a result, a classification system having a hierarchical structure is constructed. The classification frame defined by the user becomes the classification frame at the end of the hierarchical structure, and the other classification frames are set by the system. The hierarchical structure is set so that a classification error approaches a minimum when automatic classification is performed using a sample document as an input document.

  According to such processing, the classification frames are clustered into clusters as shown in FIG. 4, for example. Focusing on the low-accuracy cluster α and a cluster similar thereto, this is made into one cluster, re-clustered, and as shown in FIG. In FIG. 5, the integrated cluster is divided into three clusters by re-clustering. By repeating the above processing, a hierarchical structure classification system as shown in FIG. 6 is constructed.

  According to this embodiment, the user can easily construct a hierarchical classification system simply by preparing a flat classification frame and its sample document without considering the hierarchical relationship of the classification system. In addition, since the classification system is updated so as to improve the classification accuracy, high accuracy of document classification is guaranteed.

  In addition, this invention is not limited to the above-mentioned Example, A various change is possible. For example, the present invention may be constructed as a classification system optimization unit (learning unit) 15 as shown in FIG. That is, the classification system optimization unit 15 can optimize the classification system used in the document classification unit 11 by using a document in the classified document storage unit 12, another sample document, or the like. For example, as shown in FIG. 8, the classification system optimization unit 15 includes a document input unit 151, an evaluation unit 152, a classification system storage unit 153, a classification system change unit 154, and the like. The document input unit 151, the evaluation unit 152, the classification system storage unit 153, and the classification system change unit 154 of the classification system optimization unit 15 are the document input unit 141, the evaluation unit 144, and the classification system of the classification system hierarchization unit 14 of FIG. The operations correspond to those of the storage unit 143 and the classification system changing unit 145, and their operations are the same as those indicated by the broken lines in FIG. 3 (steps S12 to S16).

  In this modified example, when the classification accuracy of the existing classification system is lower than the reference value, or when the classification system is modified according to the situation, the classification system of the document classification unit 11 is stored in the classification system storage unit 153 and optimized. A classified document or a sample document representing the situation is input from the document input unit 151 and evaluated and changed. In this case as well, the classification accuracy is improved and the situation is matched.

It is a block diagram which shows the Example of this invention. It is a block diagram which shows the structural example of the principal part of the said Example. It is a flowchart explaining operation | movement of the said Example. It is a figure explaining operation | movement of the said Example, and is a figure explaining the state of the clustered classification frame. It is a figure explaining the operation | movement of the said Example, It is a figure explaining the state which integrated after focusing on the cluster whose precision is lower than a reference value, and then reclustered. It is a figure explaining operation | movement of the said Example, and is a figure which shows the example of the hierarchical structure classification | category system finally obtained. It is a block diagram explaining the modification of the said Example. It is a block diagram which shows the structural example of the principal part in the modification of the above-mentioned Example.

Explanation of symbols

1 Document classification device 10 Document input unit 11 Document classification unit 12 Document storage unit 13 Sample document storage unit 14 Classification system hierarchization unit 15 Classification system optimization unit 141 Sample document input unit 142 Initial classification system generation unit 143 Classification system storage unit 144 Evaluation unit 145 Classification system change unit 151 Document input unit 152 Evaluation unit 153 Classification system storage unit 154 Classification system change unit

Claims (9)

In a classification system generation device that generates a hierarchical classification system of a document classification device,
Input a sample document for each non-hierarchical terminal classification frame, cluster the non-hierarchical terminal classification frame with reference to the sample document, and use the cluster obtained as a result of clustering as a higher classification frame to An initial hierarchical structure classification system generating means for generating a hierarchical structure to be arranged above the terminal classification frame as an initial hierarchical structure classification system;
A hierarchical structure classification system storage means for storing an initial hierarchical structure classification system or a modified hierarchical structure classification system;
Classification accuracy evaluation means for classifying the sample document with reference to the hierarchical structure classification system stored in the hierarchical structure classification system storage means, and determining a higher classification frame whose classification accuracy is lower than a reference value;
When a higher classification frame whose classification accuracy is lower than the reference value is determined, the cluster of the higher classification frame whose accuracy is evaluated to be lower than the reference value by the classification accuracy evaluation means is integrated with other clusters to form one cluster The end classification frame included in the one cluster that has been reorganized and integrated is reclustered with reference to the sample document included in the end classification frame, and the cluster obtained as a result of the reclustering is converted into the one cluster. Hierarchical structure change means for changing the hierarchical structure classification system stored in the hierarchical structure classification system storage means so that the upper classification frame is lower than the upper classification frame corresponding to
When the predetermined end condition is not satisfied, the determination by the classification result evaluating means and the change of the hierarchical structure classification system by the hierarchical structure changing means are repeated, and when the predetermined end condition is satisfied, the hierarchical structure classification system A classification system generation apparatus characterized in that the hierarchical structure classification system stored in a storage means is used as a classification system of the document classification apparatus.   2. The classification system generating apparatus according to claim 1, wherein the other cluster integrated with the cluster evaluated to have the accuracy lower than the reference value is a cluster similar to the cluster evaluated to have the accuracy lower than the reference value.   The classification system generation device according to claim 1 or 2, wherein the termination condition is that any cluster is not evaluated as having a classification accuracy lower than a reference value. A document input means for inputting a document;
A hierarchical classification system generating means for generating a hierarchical classification system including the terminal classification frame and the classification frame,
In a document classification device for classifying the document with reference to the hierarchical structure classification system,
The hierarchical structure classification system generating means is:
Input a sample document for each non-hierarchical terminal classification frame, cluster the non-hierarchical terminal classification frame with reference to the sample document, and use the cluster obtained as a result of clustering as a higher classification frame to An initial hierarchical structure classification system generating means for generating a hierarchical structure to be arranged above the terminal classification frame as an initial hierarchical structure classification system;
A hierarchical structure classification system storage means for storing an initial hierarchical structure classification system or a modified hierarchical structure classification system;
Classification accuracy evaluation means for classifying the sample document with reference to the hierarchical structure classification system stored in the hierarchical structure classification system storage means, and determining a higher classification frame whose classification accuracy is lower than a reference value;
When a higher classification frame whose classification accuracy is lower than the reference value is determined, the cluster of the higher classification frame whose accuracy is evaluated to be lower than the reference value by the classification accuracy evaluation means is integrated with other clusters to form one cluster The end classification frame included in the one cluster that has been reorganized and integrated is reclustered with reference to the sample document included in the end classification frame, and the cluster obtained as a result of the reclustering is converted into the one cluster. Hierarchical structure change means for changing the hierarchical structure classification system stored in the hierarchical structure classification system storage means so that the upper classification frame is lower than the upper classification frame corresponding to
When the predetermined end condition is not satisfied, the determination by the classification result evaluating means and the change of the hierarchical structure classification system by the hierarchical structure changing means are repeated, and when the predetermined end condition is satisfied, the hierarchical structure classification system A document classification apparatus using the hierarchical structure classification system stored in a storage means as a classification system of the document classification apparatus. A document input means for inputting a document;
A hierarchical classification system storage means for storing a hierarchical classification system to be referred to when classifying documents;
Hierarchical structure adapting means for adapting the hierarchical structure classification system stored in the hierarchical structure classification system storage means,
In a document classification apparatus for classifying the document with reference to the adapted hierarchical structure classification system,
The hierarchical structure adaptation means is:
Classification accuracy evaluation means for classifying the sample document with reference to the hierarchical structure classification system stored in the hierarchical structure classification system storage means, and determining a higher classification frame whose classification accuracy is lower than a reference value;
When a higher classification frame whose classification accuracy is lower than the reference value is determined, the cluster of the higher classification frame whose accuracy is evaluated to be lower than the reference value by the classification accuracy evaluation means is integrated with other clusters to form one cluster The end classification frame included in the one cluster that has been reorganized and integrated is reclustered with reference to the sample document included in the end classification frame, and the cluster obtained as a result of the reclustering is converted into the one cluster. Hierarchical structure change means for changing the hierarchical structure classification system stored in the hierarchical structure classification system storage means so as to be an upper classification frame lower than the upper classification frame corresponding to
When the predetermined end condition is not satisfied, the determination by the classification result evaluating unit and the change of the hierarchical structure classification system by the hierarchical structure changing unit are repeated, and when the predetermined end condition is satisfied, the hierarchical structure classification system A document classification apparatus using the hierarchical structure classification system stored in a storage means as a classification system of the document classification apparatus. In a method for generating a hierarchical classification system of a document classification device,
(A) inputting a sample document for each non-hierarchical terminal classification frame;
(B) A non-hierarchical terminal classification frame is clustered with reference to the sample document, and a hierarchical structure in which a cluster obtained as a result of the clustering is arranged as a higher classification frame at a higher level than the non-hierarchical terminal classification frame, Generating as an initial hierarchical classification system;
(C) classifying the sample document with reference to an initial hierarchical classification system or a modified hierarchical classification system;
(D) determining a higher classification frame whose classification accuracy is lower than a reference value;
(E) When a higher classification frame whose classification accuracy is lower than the reference value is determined, the clusters of the higher classification frame evaluated as having a lower accuracy than the reference value are integrated with other clusters and reorganized into one cluster. The terminal classification frames included in the integrated one cluster are reclustered with reference to the sample document included in the terminal classification frame, and the cluster obtained as a result of the reclustering corresponds to the one cluster. Changing the hierarchical structure classification system stored in the hierarchical structure classification system storage means to return to the step (c) so as to be an upper classification frame lower than the upper classification frame;
(F) A method for generating a classification system, comprising a step of using a current hierarchical structure classification system as a classification system of the document classification device when a higher classification frame having a classification accuracy lower than a reference value is not determined. In a method for adapting a hierarchical classification system of a document classification device,
(A) classifying the sample document with reference to a hierarchical structure classification system;
(B) determining a higher classification frame whose classification accuracy is lower than a reference value;
(C) When a higher classification frame whose classification accuracy is lower than the reference value is determined, the cluster of the higher classification frame evaluated as having a lower accuracy than the reference value is integrated with other clusters and reorganized into one cluster. The terminal classification frames included in the integrated one cluster are reclustered with reference to the sample document included in the terminal classification frame, and the cluster obtained as a result of the reclustering corresponds to the one cluster. Changing the hierarchical structure classification system stored in the hierarchical structure classification system storage means to return to the step (a) so as to be an upper classification frame lower than the upper classification frame;
(D) a classification system adaptation method comprising a step of using a current hierarchical structure classification system as a classification system of the document classification device when a higher classification frame having a classification accuracy lower than a reference value is not determined. In a computer program for generating a hierarchical structure classification system for generating a hierarchical structure classification system of a document classification device,
(A) inputting a sample document for each non-hierarchical terminal classification frame;
(B) A non-hierarchical terminal classification frame is clustered with reference to the sample document, and a hierarchical structure in which a cluster obtained as a result of the clustering is arranged as a higher classification frame at a higher level than the non-hierarchical terminal classification frame, Generating as an initial hierarchical classification system;
(C) classifying the sample document with reference to an initial hierarchical classification system or a modified hierarchical classification system;
(D) determining a higher classification frame whose classification accuracy is lower than a reference value;
(E) When a higher classification frame whose classification accuracy is lower than the reference value is determined, the clusters of the higher classification frame evaluated as having a lower accuracy than the reference value are integrated with other clusters and reorganized into one cluster. The terminal classification frames included in the integrated one cluster are reclustered with reference to the sample document included in the terminal classification frame, and the cluster obtained as a result of the reclustering corresponds to the one cluster. Changing the hierarchical structure classification system stored in the hierarchical structure classification system storage means to return to the step (c) so as to be an upper classification frame lower than the upper classification frame;
(F) A classification used for causing a computer to execute a step of using the current hierarchical structure classification system as a classification system of the document classification device when a higher classification frame having a classification accuracy lower than a reference value is not determined. Computer program for system generation. In a computer program for hierarchical structure adaptation that adapts the hierarchical classification system of a document classification device,
(A) classifying the sample document with reference to a hierarchical structure classification system;
(B) determining a higher classification frame whose classification accuracy is lower than a reference value;
(C) When a higher classification frame whose classification accuracy is lower than the reference value is determined, the cluster of the higher classification frame evaluated as having a lower accuracy than the reference value is integrated with other clusters and reorganized into one cluster. The terminal classification frames included in the integrated one cluster are reclustered with reference to the sample document included in the terminal classification frame, and the cluster obtained as a result of the reclustering corresponds to the one cluster. Changing the hierarchical structure classification system stored in the hierarchical structure classification system storage means to return to the step (a) so as to be an upper classification frame lower than the upper classification frame;
(D) A classification that is used for causing a computer to execute a step of using the current hierarchical structure classification system as the classification system of the document classification device when a higher classification frame having a classification accuracy lower than a reference value is not determined. Computer program for system adaptation.

Images