JP4063489B2 - Document classification apparatus, document classification method, and computer-readable recording medium storing a program for causing a computer to execute the method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a document classification device that classifies an electronic document into any one of preset categories, a document classification method, and a computer-readable recording medium that records a program that causes a computer to execute the method. .
[0002]
[Prior art]
2. Description of the Related Art Document classification apparatuses that automatically classify a given classification target document (group) into any preset category using a document group called a corpus are conventionally known. In this type of device, for each of a large number of documents (corpus) having a wide variety of contents, the categories of the respective classification destinations are determined manually, and on the contrary, the characteristics of each category are reversed from the characteristics of the documents classified into each category. The feature is calculated. When a document to be classified is given, the features of the document are sequentially compared with the features of each category, and the document is classified into the category having the highest similarity.
[0003]
However, depending on the characteristics of the documents to be classified, the preset classification system may be inappropriate. Therefore, some document classification devices automatically change the classification system according to the tendency of the documents to be classified. Proposed. Examples of such conventional techniques include JP-A-07-049875 and JP-A-2000-011004.
[0004]
The document collection server system in Japanese Patent Application Laid-Open No. 07-049875 automatically connects to a plurality of information sources to acquire a new document, and checks the degree of conformity with the search condition described in advance by the user by the degree of conformity calculation. Then, a classification system is constructed from the relationship between the search conditions, and the adapted documents are classified and stored in a folder. In addition, the collection of information in each folder is monitored, and folder subdivision, integration, and structure change are automatically performed.
[0005]
In addition, the automatic information classification apparatus disclosed in Japanese Patent Laid-Open No. 2000-011004 provides a temporary category attached to the category and classifies the document there when there is a document that is mistakenly classified into a certain category. The subsequent documents are classified into a regular category or a temporary category having a higher similarity. That is, when a document exceeding the category of the existing category appears, a temporary category that is adjacent to the regular category and equivalent thereto is automatically generated. This temporary category can be upgraded to a regular category.
[0006]
[Problems to be solved by the invention]
However, among the above prior arts, Japanese Patent Application Laid-Open No. 07-049875 can only divide (subdivide) existing categories, so when a completely new field document appears, a category suitable for the document is selected. It cannot be generated. Nor can you delete unnecessary categories that have little or no document classification. Also, according to Japanese Patent Laid-Open No. 2000-011004, a category having low similarity to an existing category cannot be generated, and categories cannot be divided or deleted.
[0007]
In other words, the above prior art is vulnerable to qualitative changes in the classification target document group, and when a new document that cannot be classified into an existing category appears, the classification system does not match the characteristics of the classification target document group. There was a problem that classification could not be performed and classification accuracy was lowered.
[0008]
In order to eliminate the above-mentioned problems caused by the prior art, the present invention can always maintain a classification system suitable for the characteristics of the document group to be classified, and therefore can prevent a decrease in classification accuracy due to the change. An object of the present invention is to provide a computer readable recording medium having recorded thereon a document classification device, a document classification method, and a program for causing a computer to execute the method.
[0009]
[Means for Solving the Problems]
In order to solve the above-described problems and achieve the object, a document classification apparatus according to the present invention is a document classification apparatus that classifies an electronic document into one of preset categories. First feature vector space calculating means for calculating the feature vector space of each category from the feature vectors of each document constituting the set, a feature vector space having the same length of all vectors, and a feature vector space of the training document set And a second feature vector space calculating unit that calculates a feature vector space of an unclassifiable category from the weighted average of the feature vector space of the classification target document set, and each category calculated by the first feature vector space calculating unit Features of unclassifiable categories calculated by the feature vector space and the second feature vector space calculating means A classification in which any one of the categories or the unclassifiable categories is calculated as a classification target category of each document by comparing the vector space and the feature vector of each document constituting the classification target document set. A category calculation means, a category addition necessity judgment means for judging whether or not the frequency at which the uncategorized category is calculated as a classification destination category by the classification category calculation means exceeds a certain threshold, and the category addition necessity judgment means And a category addition recommendation means for recommending the operator to add a new category when it is determined that the frequency of calculation of the category that cannot be classified as the category to be classified exceeds a certain threshold. It is characterized by.
[0010]
According to the present invention, when the number of documents classified into the unclassifiable category increases, it is recommended that the operator add a new category.
[0011]
Further, the document classification device according to the present invention is a document classification device that classifies an electronic document into any one of preset categories, from the feature vector of each document that constitutes a training document set. Comparing a feature vector space calculating means for calculating a feature vector space of each category with a feature vector space of each category calculated by the feature vector space calculating means and a feature vector of each document constituting a classification target document set; Accordingly, a classification category calculation unit that calculates any one of the categories as a classification destination category of each document, and a frequency at which the classification category calculation unit calculates the classification destination category falls below a certain threshold. Category deletion / merging necessity determination means for determining whether there is a category and the category deletion / merging necessity A category deletion / merging recommendation unit that recommends the operator to delete or merge the category when the determination unit determines that there is a category whose classification destination and calculated frequency are below a certain threshold; , Provided.
[0012]
According to the present invention, when the number of documents classified into a certain category decreases, the operator is recommended to delete or merge the category.
[0013]
In the document classification apparatus according to the present invention, in the above-described invention, there is further provided a category division requirement for determining whether there is a category whose frequency calculated by the classification category calculation unit and the frequency calculated by the classification category calculation unit exceeds a certain threshold. When it is determined that there is a category whose frequency calculated as a classification destination category by the rejection determination means and the category division necessity determination means exceeds a certain threshold value, division of the category is recommended to the operator And a category division recommending means.
[0014]
According to the present invention, when the number of documents classified into a certain category increases, the operator is recommended to divide the category.
[0015]
The document classification method according to the present invention is a document classification method for classifying an electronic document into any one of preset categories, from the feature vector of each document constituting a training document set. A first feature vector space calculating step for calculating a feature vector space of each category, a feature vector space in which all vectors have the same length, a feature vector space of a training document set, and a feature vector space of a classification target document set A second feature vector space calculating step of calculating a feature vector space of an unclassifiable category from a weighted average, a feature vector space of each category calculated in the first feature vector space calculating step, and the second feature vector space The feature vector space of the unclassifiable category calculated in the calculation process and each of the documents constituting the classification target document set A classification category calculation step of calculating any one of the categories or the unclassifiable categories as a classification destination category of each document by comparing with a feature vector of the document, and classification in the classification category calculation step A category addition necessity determination step for determining whether or not the impossible category is a classification destination category and whether the frequency of calculation exceeds a certain threshold value, and the category addition necessity determination step calculates the classification impossible category as a classification destination category. A category addition recommendation step of recommending the operator to add a new category when it is determined that the frequency exceeds a certain threshold value.
[0016]
According to the present invention, when the number of documents classified into the unclassifiable category increases, it is recommended that the operator add a new category.
[0017]
The document classification method according to the present invention is a document classification method for classifying an electronic document into any one of preset categories, from the feature vector of each document constituting a training document set. Comparing a feature vector space calculating step of calculating a feature vector space of each category with a feature vector space of each category calculated in the feature vector space calculating step and a feature vector of each document constituting a classification target document set; The classification category calculation step of calculating any one of the categories as the classification destination category of each document, and the frequency calculated as the classification destination category in the classification category calculation step falls below a certain threshold. Category deletion / merging necessity determination process for determining whether there is a category and the category deletion / merging necessity determination process When it is determined that there is a category whose classification frequency is lower than a certain threshold value in the category, the category deletion / merging recommendation process that recommends the operator to delete or merge the category is It is characterized by including.
[0018]
According to the present invention, when the number of documents classified into a certain category decreases, the operator is recommended to delete or merge the category.
[0019]
Further, the document classification method according to the present invention is the above-described invention, further comprising a category division requirement for determining whether or not there is a category whose frequency calculated with the classification category in the classification category calculation step exceeds a certain threshold. When it is determined that there is a category in which the frequency calculated as a classification target category in the rejection determination step and the category division necessity determination step exceeds a certain threshold value, division of the category is recommended to the operator And a category division recommendation process.
[0020]
According to the present invention, when the number of documents classified into a certain category increases, the operator is recommended to divide the category.
[0021]
Further, the recording medium according to the present invention records a program for causing a computer to execute the method, so that the program can be read by the computer, and thus the method can be performed by the computer. .
[0022]
DETAILED DESCRIPTION OF THE INVENTION
Exemplary embodiments of a document classification device, a document classification method, and a computer-readable recording medium recording a program for causing a computer to execute the method will be described below in detail with reference to the accompanying drawings.
[0023]
(Embodiment)
First, the hardware configuration of the document classification apparatus according to the embodiment of the present invention will be described. FIG. 1 is an explanatory diagram showing a hardware configuration of a document classification apparatus according to an embodiment of the present invention. In the figure, 101 indicates a CPU that controls the entire system, 102 indicates a ROM that stores basic input / output programs, and 103 indicates a RAM that is used as a work area of the CPU 101.
[0024]
Reference numeral 104 denotes an HDD (hard disk drive) that controls reading / writing of data with respect to the HD (hard disk) 105 according to the control of the CPU 101, and 105 denotes an HD that stores data written according to the control of the HDD 104. Yes. Reference numeral 106 denotes an FDD (floppy disk drive) that controls reading / writing of data with respect to the FD (floppy disk) 107 under the control of the CPU 101, and 107 denotes a removable FD that stores data written according to the control of the FDD 106. Respectively.
[0025]
Reference numeral 108 denotes a cursor, menu, window, or display for displaying various data such as characters and images. Reference numeral 109 denotes a network that is connected to the network NET via the communication line 110 and functions as an interface between the network NET and the CPU 101. Each board is shown. Reference numeral 111 denotes a keyboard having a plurality of keys for inputting characters, numerical values, and various instructions, and 112 denotes a mouse for selecting and executing various instructions, selecting a processing target, moving a cursor, and the like. ing.
[0026]
Reference numeral 113 denotes a scanner that optically reads characters and images, 114 denotes a printer that prints characters and images according to the control of the CPU 101, 115 denotes a CD-ROM that is a removable recording medium, and 116 denotes a CD-ROM 115. Reference numeral 100 denotes a CD-ROM drive for controlling the reading of data with respect to the above.
[0027]
Next, a functional configuration of the document classification device according to the embodiment of the present invention will be described. FIG. 2 is an explanatory diagram functionally showing the configuration of the document classification apparatus according to the embodiment of the present invention.
[0028]
A document storage unit 201 holds a document group (classification target document set) 201a to be classified by a classification category estimation unit 203 described later. Each document is acquired from another information processing apparatus on the network via the communication line 110, input by the keyboard 111, taken from a paper medium document via the scanner 113, or CD-ROM 115. Or read from various recording media. In addition, there may be a difference when each document is acquired / accumulated (of 50 documents, 10 are acquired / accumulated one week ago, 25 are acquired / accumulated yesterday, and 15 are acquired today)・ Accumulated, etc.)
[0029]
A feature vector space estimation unit 202 holds a training document set 202a and a document-category correspondence table 202b in an initial state. The training document set 202a is a so-called corpus, and is a document set including documents belonging to various fields, for example, a document set consisting of newspaper articles for one year. The document-category correspondence table 202b is a table that stores categories in which the document is classified in association with each document in the training document set 202a. The names of the categories and the upper / lower relationships are defined in a category system table 204a of the category system management unit 204 described later.
[0030]
The feature vector space estimation unit 202 creates a classification database 202c from the training document set 202a and the document-category correspondence table 202b. FIG. 3 is an explanatory diagram for specifically explaining a method of creating the classification database 202c. The training document set 202a is made up of 10 documents D1 to D10 shown in FIG. 3A, and these documents are shown in the document-category correspondence table 202b in FIG. It shall be classified into one of the categories.
[0031]
The feature vector space estimation unit 202 first creates, for each document in the training document set 202a, a vector (feature vector) that approximately represents the semantic content of the document. There are various conventional techniques for creating feature vectors. Here, for convenience of explanation, the frequency of appearance of all words (all vocabulary, excluding unnecessary words, etc.) appearing in the document set in each document ( The simplest method is adopted in which the number of occurrences) is counted to obtain each element value (length of each vector) in the vector.
[0032]
When there are five types of words W1 to W5 that appear in the training document set 202a, the feature vectors created for each document include the appearance frequency of the word W1, the appearance frequency of W2, and the W3 It becomes a 5-dimensional vector having the appearance frequency, the appearance frequency of W4, and the appearance frequency of W5 as vector lengths. FIG. 3C shows an example of a feature vector created for each document. For example, the feature vector of the document D1 in which the word W1 appears 0 times, W2 2 times, W3 1 time, W4 0 times, and W5 3 times is V1 = (0, 2, 1, 0, 3). is there. Similarly, the feature vector of the document D5 is V5 = (1, 5, 1, 0, 4), and the feature vector of the document D8 is V8 = (0, 2, 0, 0, 2).
[0033]
After creating the feature vector for each document in the training document set 202a, the feature vector space estimation unit 202 calculates the feature vector space of each category by averaging the feature vectors of the documents classified into each category. To do. For example, the feature vector space VA of category A is obtained by averaging feature vectors of documents D1, D5, and D8 belonging to category A, and VA = (0 + 1 + 0/3, 2 + 5 + 2/3, 1 + 1 + 0/3, 0 + 0 + 0/3, 3 + 4 + 2 / 3), that is, VA = (0.33, 3, 0.66, 0, 3). Then, a classification database 202c shown in FIG. 3D is created from this VA and VB and VC obtained in the same manner.
[0034]
Here, in addition to A, B, and C, the category “classification not possible” is provided in the classification database 202c. The feature vector space of this unclassifiable category is calculated according to the following procedures (1) to (4).
[0035]
(1) Calculation of feature vector space Vα in which all vectors have the same length
First, for each document in the classification target document set 201a held in the document storage unit 201, the feature vector space estimation unit 202 includes all the words that appear in the document set, or some words selected in advance. A feature vector having an appearance frequency in each document as an element value is created. Here, the classification target document set 201a is composed of five documents D'1 to D'5 shown in FIG. 4A, and a feature vector as shown in FIG. 4B is created for each document. And
[0036]
Then, the feature vector space estimation unit 202 determines the total number of times that the words W1 to W7 appear in the training document set 202a and the classification target document set 201a (which may be referred to as the total vector length, here 87 + 59 = 146). Then, dividing by the total number of documents (here 10 + 5 = 15), the average number of word appearances per document (here, 146/15 = 9.73) is calculated. Further, this is divided by the total number of vocabularies (the number of vectors. Here, 5), and the average number of appearances of each word per document (here, 9.73 / 5 = 1.95) is obtained. calculate. A feature vector having this value as the length of each vector corresponding to the words W1 to W7 = (1.95, 1.95, 1.95, 1.95, 1.95, 1.95, 1.95). A space Vα represented by
[0037]
(2) Calculation of feature vector space Vβ of training document set 202a
Next, the feature vector space estimation unit 202 calculates the average number of times each of the words W1 to W7 appears per document in the training document set 202a, and the feature vector space of the training document set 202a. Create Vβ. Here, the feature vectors representing the feature vector space Vβ are (12/10, 11/10, 13/10, 17/10, 34/10, 0/10, 0/10), that is, (1.2, 1 .1, 1.3, 1.7, 3.4, 0, 0).
[0038]
(3) Calculation of feature vector space Vγ of classification target document set 201a
Further, the feature vector space estimation unit 202 calculates the average number of times each of the words W1 to W7 appears per document in the classification target document set 201a, and the feature of the classification target document set 201a. A vector space Vγ is created. Here, the feature vectors representing the feature vector space Vγ are (7/5, 6/5, 8/5, 10/5, 18/5, 10/5, 0/5), that is, (1.4, 1 .2, 1.6, 2, 3.6, 2, 0).
[0039]
(4) Calculation of feature vector space V of unclassifiable category
Then, the feature vector space estimation unit 202 calculates the feature vector space V of the category that cannot be classified by taking the weighted average of Vα, Vβ, and Vγ calculated above. For example, if the weight given to Vα is 0.15, the weight given to Vβ is 0.45, and the weight given to Vγ is 0.40, the feature vector representing the feature vector space V is ((1.95 × 0 .15) + (1.2 × 0.45) + (1.4 × 0.40), (1.95 × 0.15) + (1.1 × 0.45) + (1.2 × 0) .14), ...), that is, (1.39, 1.27, 1.52, 1.86, 3.26, 1.09, 0.29). Then, this V is registered in the classification database 202c as a feature vector space of an unclassifiable category.
[0040]
In this way, the feature vector space V of the category that cannot be classified is divided into “feature vector space Vα having the same length of all vectors”, “feature vector space Vβ of training document set 202a”, and “feature vector space of classification target document set 201a”. The meaning of the estimation by the weighted average of “Vγ” is as follows.
[0041]
First, a feature vector space Vα in which all vectors have the same length plays a role of assuming a feature vector space of an unknown category. In the unknown category, since it is unknown what distribution each vocabulary takes, a vector space in which the appearance frequencies of all vocabularies are the same is assumed by this Vα.
[0042]
However, this Vα is not a natural language space expression. For example, “the” in English has a high frequency of appearance in any document regardless of the category, so it is natural that the frequency is high even in a category that cannot be classified. Therefore, by considering the feature vector space Vβ of the training document set 202a, the vector length of a word whose appearance frequency does not depend on the category (W5 in the example of FIG. 3) can be improved.
[0043]
Taking the feature vector space Vγ of the classification target document set 201a into consideration is basically the same as described above. However, since the training document set 202a has a limit on the scale (because the contents of each document must be examined in advance and a category must be manually assigned), Vγ in addition to Vβ is added to supplement the scale. Consider. In addition, when Vβ is not suitable for a document set to be classified, such as when the training document is collected and when the vocabulary pattern changes, the document to be classified is considered by considering Vγ. Can be optimized to match For example, as shown in FIG. 3D, in the feature vector spaces VA to VC of the categories A to C, the vector lengths of W6 and W7 are both 0, but in the feature vector space V of the non-classifiable category, 1. Since it is 09, 0.29, W6 or W7, that is, a document including a new word that has not appeared in the training document set 202a, a word in a new field, or the like can be classified into the unclassifiable category. Increases nature.
[0044]
In the above description, Vα and Vγ are calculated using all the documents in the classification target document set 201a. However, when the number is large, the processing load is large. A part of the extracted document may be used.
[0045]
Note that the feature vector space estimation unit 202 corresponds to “first feature vector space calculation means”, “second feature vector space calculation means”, or “feature vector space calculation means” in the claims, and processing to be performed Includes “first feature vector space calculating step”, “second feature vector space calculating step” or “feature vector space calculating step”.
[0046]
Returning to FIG. 2, description of the remaining functional units will be continued. Reference numeral 203 denotes a classification category estimation unit, in which the feature vector of each document of the classification target document set 201a is stored in the feature vector space of each category calculated by the feature vector space estimation unit 202 (this is stored in the classification database 202c). The degree of similarity is calculated sequentially. There are many conventional techniques for calculating the degree of similarity, such as TF / IDF, native Bayes, least squares, and maximum entropy, and any of them may be adopted here.
[0047]
Then, the category having the highest similarity is estimated as the classification destination of the document, and the classification result list 203a as illustrated in FIG. 5 is created by associating the document with the classification destination category. The classification category estimation unit 203 corresponds to “classification category calculation means” in the claims, and the processing performed by the classification category estimation unit 203 corresponds to “classification category calculation step” in the claims.
[0048]
A category system management unit 204 holds a category system table 204a and a category system change rule 204b in advance. In the category system table 204a, the name of each category into which the document is classified and its upper / lower relationship are defined. As long as the names and relationships of the categories are known, the data structure of the category system table 204a may be anything.
[0049]
The category system change rule 204b is a plurality of categories for determining whether it is desirable to add a new category to the category system table 204a or to divide / delete / merge an already registered category. Rules (judgment criteria).
[0050]
For example, with regard to category addition, “(a-1) The frequency at which it is determined that classification is not possible has exceeded a threshold (specifically,“ the percentage of documents that are determined to belong to the category that cannot be classified among all documents is 20%. There is a rule that “exceeded 30 or more documents determined to belong to an unclassifiable category”). If this rule falls under this rule, it is determined that it is desirable to add a new category.
[0051]
The rules for adding a category include “(a-2) the number of documents belonging to an unclassifiable category exceeds a threshold”, “(a-3) an evaluation function using the above frequency and / or the number of documents. The result exceeded the threshold (specifically, “the number of consecutive appearances of documents determined to belong to the unclassifiable category exceeded a value corresponding to 15% of the total number of documents”). Below, among these rules, the first mentioned “the ratio of documents determined to belong to the unclassifiable category in all documents exceeds 20%” is adopted as the category addition rule.
[0052]
Further, as a rule regarding category division, for example, “(b-1) The frequency at which a certain category is determined to be a classification destination exceeds a threshold (specifically,“ all documents are determined to belong to a certain category ”). The number of documents that belong to a certain category has exceeded the threshold value, such as “the document that has been determined to belong to a certain category has appeared more than 30 consecutively”), “(b-2) "(B-3) The result of the evaluation function using the frequency and / or the number of documents exceeds a threshold (specifically," the number of consecutive appearances of a document determined to belong to a certain category is Etc.) ”, etc., which exceeded 15% of the total number of documents. Below, among these rules, the first mentioned “the ratio of documents determined to belong to a certain category in all documents exceeds 20%” is adopted as the category division rule.
[0053]
Further, as a rule regarding category deletion or merging, “(c-1) The frequency at which a certain category is determined to be a classification destination falls below a threshold (specifically,“ determined that it belongs to a certain category in all documents ”). The ratio of the selected documents is less than 5%, “50 or more documents classified in a category other than a certain category appear continuously”, etc.). Here, the first mentioned “the ratio of documents determined to belong to a certain category to less than 5% in all documents” is adopted as the category deletion / merging rule.
[0054]
When the classification system estimation unit 203 finishes classifying the document, the category system management unit 204 refers to the created classification result list 203a to determine whether it is desirable to add, divide, delete, or merge categories. Determines whether an event corresponding to each rule of the category system change rule 204b has occurred.
[0055]
For example, the classification result list 203a is as shown in FIG. 5, and the category system change rule 204b is one of the additional rules “the ratio of documents determined to belong to the unclassifiable category exceeds 20% in all documents”. 5, it can be seen from FIG. 5 that the number of documents in the uncategorized category (here, 2) exceeds 20% of the total number of documents (here, 5), so it is determined that the addition of a new category is desirable. To do. Whether categories are divided or deleted / merged is also determined in the same manner as described above.
[0056]
The category system management unit 204 corresponds to “category addition necessity determination means”, “category division necessity determination means” or “category deletion / merger necessity determination means” in the claims, and the processing to be performed It includes a “category addition necessity determination step”, “category division necessity determination step” or “category deletion / merger necessity determination step” in the claims.
[0057]
The user interface unit 205 reads the classification result list 203a created by the classification category estimation unit 203 and the category system table 204a of the category system management unit 204, and the category tree structure and the titles of documents classified into the categories. Are displayed graphically on the screen. In addition, when the category system management unit 204 determines that addition, division, deletion, or merging of categories is desirable, a dialog that recommends the operator to perform addition of a category, etc., and parameters necessary for the execution (addition) Display a dialog for entering the category name). These display examples will be described later.
[0058]
The user interface unit 205 corresponds to the “category addition recommendation means”, “category division recommendation means” or “category deletion / merging recommendation means” described in the claims. “Category addition recommendation process”, “Category division recommendation process” or “Category deletion / merging recommendation process” are included.
[0059]
The document storage unit 201, the feature vector space estimation unit 202, the category category estimation unit 203, the category system management unit 204, and the user interface unit 205 are recorded in a recording medium such as the ROM 102, the RAM 103 or the hard disk 105, and the floppy disk 107, respectively. The function of each unit is realized by the CPU 101 or the like executing instruction processing according to the instructions described in the program.
[0060]
Next, a procedure for document classification processing of the document classification device according to the embodiment of the present invention will be described. FIG. 6 is a flowchart showing a procedure of document classification processing of the document classification device according to the embodiment of the present invention. When a new document is added to the classification target document set 201a or when an operator instructs the classification of the document, the processing according to this flowchart is started.
[0061]
In step S601, the feature vector space estimation unit 202 creates a feature vector for each document in the training document set 202a. In step S602, document feature vectors are averaged for each category in the document-category correspondence table 202b to calculate a feature vector space for each category.
[0062]
If the processing according to this flowchart has been performed before this (that is, in the case of the second and subsequent classification processing), the feature vector space of each category based on the training document set 202a is also used during the previous processing. Since the calculation is performed and the result is stored in the classification database 202c, the processes in steps S601 and S602 can be omitted.
[0063]
In step S603, the feature vector space V of the category that cannot be classified is estimated. As described above, this is because the feature vector space Vα (calculated from both the classification target document set 201a and the training document set 202a) having the same length of all vectors, the feature vector space Vβ of the training document set 202a, It is calculated by a weighted average with the feature vector space Vγ of the classification target document set 201a. In step S604, the feature vector space of each category calculated in steps S602 and S603 is stored in the classification database 202c.
[0064]
In step S605, the classification category estimation unit 203 creates a feature vector for each document in the classification target document set 201a. If Vα and Vγ are calculated using all of the classification target documents in step S603, the feature vectors of all the documents have already been created at that time, and therefore the processing in step S605 may be omitted. it can. If Vα and Vγ are calculated using a part of the classification target document in step S603, a feature vector may be created only for a document that is not included therein.
[0065]
In step S606, the classification category estimation unit 203 sequentially compares the feature vector of each document created in step S605 with the feature vector space of each category in the classification database 202c created in step S604, and is most similar. A category having a high degree is estimated as a classification destination of the document. In step S607, each document and its classification destination category are stored in the classification result list 203a.
[0066]
In step S608, the user interface unit 205 reads the classification result list 203a created in step S607 and displays a classification result list window as shown in FIG. Then, the process according to this flowchart is terminated.
[0067]
Next, the procedure of the category addition process of the document classification device according to the embodiment of the present invention will be described. FIG. 8 is a flowchart showing the procedure of the category addition process of the document classification device according to the embodiment of the present invention.
[0068]
Immediately after the document classification process shown in FIG. 6 is completed, the process according to this flowchart is started (or only this process may be performed periodically regardless of the classification process). It is assumed that the classification result list window displayed on the screen at the start time is as shown in FIG. This figure shows a state in which a large number of documents (for example, more than 20% of the total number of documents) are classified in the unclassifiable category.
[0069]
In step S801, the category system management unit 204 refers to the classification result list 203a created in step S607 of FIG. 6 to determine whether an event corresponding to the category addition rule of the category system change rule 204b has occurred. judge. Then, when an event corresponding to the category addition rule has occurred (Yes at Step S801), the process proceeds to Step S802. When no event has occurred (No at Step S801), the processing according to this flowchart is terminated as it is.
[0070]
In step S802, the user interface unit 205 displays a change recommendation dialog as shown in FIG. Here, it is recommended to add a new category. In step S803, it waits for any button to be clicked with the mouse, and when it is the change start button 901 that the mouse is clicked (Yes in step S803), the process proceeds to step S804. If the cancel button 902 is clicked by the mouse (No at step S803), the processing according to this flowchart is terminated as it is.
[0071]
In step S804, the user interface unit 205 deletes the change recommendation dialog displayed in step S802 and displays a category addition dialog as shown in FIG. This is a dialog for allowing the operator to input the name of a new category to be added. In step S805, it waits for any button to be clicked with the mouse, and when the mouse click is on the “forward” button 1001 (Yes in step S805), the process proceeds to step S807, and the “forward” button 1001 is displayed. If not (No in step S805), the process proceeds to step S806, and processing corresponding to the other clicked button is performed.
[0072]
In step S807, the category addition dialog displayed in step S804 is deleted and a document allocation dialog as shown in FIG. 11 is displayed. This is a dialog for allowing the operator to specify a document to be classified into a new category to be added (in this case, an information home appliance category), and displays a list of all documents classified in the unclassifiable category. In step S808, it waits for any button to be clicked with the mouse, and when the mouse click is for the “complete” button 1101 (Yes in step S808), the process proceeds to step S810, and the “complete” button 1101 is displayed. If not (No in step S808), the process proceeds to step S809, and processing corresponding to the other clicked button is performed.
[0073]
In step S810, the user interface unit 205 sends the category name (in this case, “information home appliance”) input to the category addition dialog when the “forward” button 1001 is clicked in step S805 to the category system management unit 204. Notify them. In response to this, the category system management unit 204 adds a new category “information home appliance” to the category system table 204a.
[0074]
In step S811, the user interface unit 205 selects the document selected at the time when the “complete” button 1101 in the document assignment dialog is clicked in step S808 (here, all the documents in the unclassifiable category have been selected). To the feature vector space estimation unit 202. Receiving this, the feature vector space estimation unit 202 calculates the feature vector space of the category to be added by acquiring the feature vectors created in step S605 of FIG. 6 for the notified document and taking the average of them. To do. Then, this is associated with the category name and registered in the classification database 202c.
[0075]
Further, in step S812, after the document assignment dialog displayed in step S807 is deleted, the process proceeds to step S606 in FIG. 6, and the classification of the classification target document set 201a is performed again based on the updated classification database 202c. Execute. FIG. 12 is an explanatory diagram illustrating an example of a classification result list window immediately after a new category “information home appliance” is added. The documents classified in the unclassifiable category in FIG. 7 are classified in the information appliance category in FIG.
[0076]
Next, the procedure of category division processing of the document classification device according to the embodiment of the present invention will be described. FIG. 13 is a flowchart showing the procedure of category division processing of the document classification device according to the embodiment of the present invention. Immediately after the category addition process shown in FIG. 8 is completed, the process according to this flowchart is started. It is assumed that the classification result list window displayed on the screen at the start time is as shown in FIG. This figure shows a state where a large number of documents (for example, more than 20% of the total number of documents) are classified in the portable terminal category.
[0077]
In step S1301, the category system management unit 204 refers to the classification result list 203a updated in step S811 in FIG. 8 to determine whether an event corresponding to the category division rule of the category system change rule 204b has occurred. judge. When an event corresponding to the category division rule has occurred (Yes at Step S1301), the process proceeds to Step S1302, and when it has not occurred (No at Step S1301), the processing according to this flowchart is terminated.
[0078]
In step S1302, the user interface unit 205 displays a change recommendation dialog as shown in FIG. Here, the division of the mobile terminal category is recommended. In step S1303, it waits for any button to be clicked with the mouse, and when it is the change start button 1501 that the mouse is clicked (Yes at step S1303), the process proceeds to step S1304. If the cancel button 1502 is clicked by the mouse (No at Step S1303), the processing according to this flowchart is terminated as it is.
[0079]
In step S1304, the user interface unit 205 deletes the change recommendation dialog displayed in step S1302, and displays a category division dialog as shown in FIG. This is a dialog for allowing the operator to input the name of each category after division. In step S1305, it waits for any button to be clicked with the mouse, and when the mouse click is on the “forward” button 1601 (Yes in step S1305), the process proceeds to step S1307, and the “forward” button 1601 is displayed. If not (No at step S1305), the process proceeds to step S1306, and processing corresponding to the other clicked button is performed.
[0080]
In step S1307, the category division dialog displayed in step S1304 is deleted, and a document allocation dialog as shown in FIG. 17 is displayed. This is a dialog that allows the operator to specify a document to be classified in one of the categories after division (here, the mobile phone category). All of the categories classified in the category before division (here, the mobile device category) A list of documents is displayed.
[0081]
In step S1308, it waits for any button to be clicked with the mouse, and when the mouse click is the “forward” button 1701 (Yes in step 1308), the process proceeds to step S1310, and the “forward” button 1701 is displayed. If not (No in step S1308), the process proceeds to step S1309, and processing corresponding to the other clicked button is performed.
[0082]
In step S1310, the document allocation dialog displayed in step S1307 is deleted, and a similar document allocation dialog is displayed for the other category (in this case, the mobile PC category) after the division. However, in this dialog, a list of documents obtained by excluding the documents classified in the other categories after the division in step S1308 from all the documents in the categories before the division is displayed. In step S1311, it waits for any button to be clicked with the mouse, and when it is the “complete” button 1702 (YES in step 1311), the process proceeds to step S1313, and the “complete” button 1702 is displayed. If not (No in step S1311), the process proceeds to step S1312, and processing corresponding to the other clicked button is performed.
[0083]
In step S1313, the user interface unit 205 inputs the category name before division (here, “portable terminal”) and the category name after division (in step S1305, when the “forward” button 1601 is clicked) to the category division dialog. The category name (in this case, “mobile phone” “mobile PC”) is notified to the category system management unit 204. Receiving this, the category system management unit 204 deletes the category “mobile terminal” from the category system table 204a and additionally writes new categories “mobile phone” and “mobile PC”.
[0084]
In step S1314, the user interface unit 205 notifies the feature vector space estimation unit 202 of the category names before and after the division and the documents assigned to the respective categories after the division in steps S1308 and S1311. Receiving this, the feature vector space estimation unit 202 deletes the feature vector space of the category before division from the classification database 202c, calculates the feature vector space of each category after division, and associates it with the category name. Register in the above database.
[0085]
In step S1315, the document allocation dialog displayed in step S1310 is deleted, and then the process proceeds to step S606 in FIG. 6 to reclassify the classification target document set 201a based on the updated classification database 202c. Execute. FIG. 18 is an explanatory diagram illustrating an example of a classification result list window immediately after the mobile terminal category is divided into the mobile phone category and the mobile PC category.
[0086]
A part of the document classified into the portable terminal category in FIG. 14 is classified into the portable PC category in FIG. As a result of classifying all the documents of the classification target document set 201a again after the category division, documents that are not included in the portable terminal category before the division (for example, “Company I proposes a voice input standard for portable PCs. “Company J agrees”) is included in the mobile PC category after the division. Although illustration is omitted, the mobile phone category is the same as described above. In the above description, the category is divided into two categories, but the number may be any number.
[0087]
Next, the procedure of category deletion / merging processing of the document classification device according to the embodiment of the present invention will be described. FIG. 19 is a flowchart showing the procedure of category deletion / merging processing of the document classification device according to the embodiment of the present invention. Immediately after the category dividing process shown in FIG. 13 is completed, the process according to this flowchart is started. It is assumed that the classification result list window displayed on the screen at the start time is as shown in FIG. This figure shows a state where there are few documents classified into the iNet dedicated terminal category (for example, less than 5% of the total number of documents).
[0088]
In step S1901, the category system management unit 204 refers to the classification result list 203a updated in step S1314 in FIG. 13 and has an event corresponding to the category deletion / merging rule of the category system change rule 204b occurred? Determine if. If an event corresponding to the category deletion / merging rule has occurred (Yes at Step S1901), the process proceeds to Step S1902, and if not (No at Step S1901), the process according to this flowchart is terminated.
[0089]
In step S1902, the user interface unit 205 displays a change recommendation dialog as shown in FIG. Here, it is recommended to delete or merge the iNet dedicated terminal category. In step S1903, it waits for any button to be clicked with the mouse, and when it is the change start button 2101 that has been clicked with the mouse (Yes at step S1903), the process proceeds to step S1904. If the cancel button 2102 is clicked by the mouse (No at Step S1903), the processing according to this flowchart is terminated as it is.
[0090]
In step S1904, the user interface unit 205 deletes the change recommendation dialog displayed in step S1902, and displays a category deletion / merger dialog as shown in FIG. This is a dialog for allowing the operator to specify whether to delete or merge categories with a small number of documents. Note that the “forward” button 2201 is grayed out while deletion is specified, and the “complete” button 2202 is grayed out so that it cannot be pressed while merging is specified.
[0091]
Then, it waits for any button to be pressed in step S1905, and if it is the “complete” button 2202 (Yes in step S1905), the deletion of the category is designated, so the category to be deleted (In this case, “iNet dedicated terminal”) is notified to the category system management unit 204 in step S1906 and is deleted from the category system table 204a. In step S1907, the feature vector space estimation unit 202 is also notified and deleted from the document-category correspondence table 202b and the classification database 202c. In step S1908, the category deletion / merging dialog displayed in step S1904 is deleted, and the process proceeds to step S606 in FIG.
[0092]
If the “complete” button 2202 is not pressed in step S 1905 (No in step S 1905), the process proceeds to step S 1909, and it is determined whether or not it is the “forward” button 2201. If the button is not the “forward” button 2201 (No in step S1909), the process is performed according to the other button pressed in step S1910. If the button is the “forward” button 2201 (Yes in step S1909) The process proceeds to step S1911.
[0093]
In this case, since merging of categories is designated, in step S1911, the category deletion / merging dialog displayed in step S1904 is deleted and a merging destination selection dialog as shown in FIG. 23 is displayed. This is a dialog for allowing the operator to specify in which other category a document of a category deleted by merging is classified, and all categories other than the merged category are displayed in a list.
[0094]
Then, it waits for any button to be pressed in step S1912, and if it is not the “forward” button 2301 (No in step 1912), it corresponds to the other button pressed in step S1913. The process is performed, but if it is the “forward” button 2301 (Yes in step S1912), the process proceeds to step S1914.
[0095]
In step S1914, the merge destination selection dialog displayed in step S1911 is deleted, and a document allocation dialog as shown in FIG. 24 is displayed. This is because any of the documents in the category to be deleted by merging is selected as the categorization destination category (the category selected when the “Proceed” button 2301 is pressed in step S1912. Here, it is the information appliance category. A list of all documents in the categories to be merged is displayed.
[0096]
Then, it waits for any button to be pressed in step S1915, and if it is not the “complete” button 2401 (No in step 1915), it corresponds to the other button pressed in step S1916. The process is performed, but if it is the “complete” button 2401 (Yes in step S1915), the process proceeds to step S1917.
[0097]
In step S1917, the user interface unit 205 notifies the category system management unit 204 of the name of the category to be merged (here, “iNet dedicated terminal”), and deletes it from the category system table 204a. In step S1918, the category names of the merge source and merge destination, and the document assigned to the merge destination (the document selected when the “complete” button 2401 was pressed in step S1915) are converted into the feature vector space estimation unit. 202 is notified. Upon receiving this, the feature vector space estimation unit 202 rewrites the merging source category of the document-category correspondence table 202b with the merging destination category, recalculates the feature vector space of the merging destination category, and registers it in the classification database 202c. Then, the feature vector space of the category to be merged is deleted from the database.
[0098]
In step S1919, the document allocation dialog displayed in step S1914 is deleted, and then the process proceeds to step S606 in FIG. 6 to reclassify the classification target document set 201a based on the updated classification database 202c. Execute. FIG. 25 is an explanatory diagram illustrating an example of a classification result list window immediately after the iNet dedicated terminal category is deleted or merged with the information appliance category.
[0099]
As described above, according to this embodiment, the number of documents in a specific category (including unclassifiable categories) does not increase too much every time document classification is performed by adding documents or by document classification instructions. It is automatically checked whether there are too few documents in a specific category, and on the contrary, it is recommended to add, divide, delete, or merge categories based on the check result. By simply inputting necessary items according to recommendations, it is possible to always maintain an optimal classification system that matches the characteristics of the document to be classified (ease of maintenance of the classification system). Therefore, even when there is a qualitative change such as a change in the field trend of the classification target document, it is possible to always obtain an appropriate classification result.
[0100]
Further, the feature vector space V of the unclassifiable category is obtained by weighted averaging of the feature vector space Vα having the same length of all vectors, the feature vector space Vβ of the training document set 202a, and the feature vector space Vγ of the classification target document set 201a. Therefore, even when a document having a low similarity with an existing category appears, it is possible to generate a category in a completely new field and classify it.
[0101]
The document classification methods (including the category addition method, category division method, and category deletion / merging method) described in the above embodiment are executed by executing a prepared program on a computer such as a personal computer or a workstation. Realized. This program is recorded on a computer-readable recording medium such as a hard disk, floppy disk, CD-ROM, MO, and DVD, and is read from the recording medium and executed by the computer. Further, this program can be distributed via the recording medium and a network such as the Internet.
[0102]
【The invention's effect】
As described above, the document classification device according to the present invention is a document classification device that classifies an electronic document into any one of preset categories. The feature vector space of each category is calculated from the feature vectors of each document constituting the document set for use, and the second feature vector space calculating means includes a feature vector space in which all vectors have the same length, a training document set A feature vector space of an unclassifiable category is calculated from the weighted average of the feature vector space and the feature vector space of the classification target document set, and a classification category calculating unit calculates each category calculated by the first feature vector space calculating unit. Features of unclassifiable categories calculated by the feature vector space and the second feature vector space calculating means By comparing the vector space and the feature vector of each document constituting the classification target document set, any one of the categories or the non-classifiable categories is calculated as a classification destination category of the documents, A category addition necessity determination unit determines whether the frequency that the uncategorized category is calculated as a classification destination category by the classification category calculation unit exceeds a certain threshold, and the category addition recommendation unit determines whether the category addition is necessary If it is determined by the determination means that the frequency of calculating the category that cannot be classified as the category to be classified exceeds a certain threshold, the operator is recommended to add a new category. If the number of documents increases, the operator is encouraged to add a new category, which matches the characteristics of the group of documents to be classified. Classification system can always be maintained, thus there is an effect that the document classification apparatus can be obtained which can prevent a decrease in classification accuracy due to the change.
[0103]
The document classification device according to the present invention is a document classification device for classifying an electronic document into any one of preset categories, wherein the feature vector space calculation means constitutes a training document set. The feature vector space of each category is calculated from the feature vector of each document, and the classification category calculation means calculates the feature vector space of each category calculated by the feature vector space calculation means and each document constituting the classification target document set. By comparing the feature vector, any one of the categories is calculated as a classification destination category of each document, and the category deletion / merging necessity determination unit determines whether the classification destination is determined by the classification category calculation unit. Determine whether there is a category whose category and frequency calculated below a certain threshold, and recommend category deletion / merging However, when it is determined that there is a category whose frequency calculated as the category to be classified by the category deletion / merging necessity determination unit falls below a certain threshold, the deletion or merging of the category is performed for the operator. Therefore, if there are fewer documents classified into a certain category, it is recommended that the operator delete or merge the category, thereby maintaining a classification system consistent with the characteristics of the group of documents to be classified. Therefore, it is possible to obtain a document classification device capable of preventing a decrease in classification accuracy due to the change.
[0104]
In the document classification apparatus according to the present invention, in the above-described invention, there is further provided a category division requirement for determining whether there is a category whose frequency calculated by the classification category calculation unit and the frequency calculated by the classification category calculation unit exceeds a certain threshold. When it is determined that there is a category whose frequency calculated as a classification destination category by the rejection determination means and the category division necessity determination means exceeds a certain threshold value, division of the category is recommended to the operator Category division recommending means, so that when the number of documents classified into a certain category increases, the operator is recommended to divide the category, and accordingly, a classification system suitable for the characteristics of the classification target document group Therefore, it is possible to obtain a document classification apparatus that can always maintain the image quality, and therefore can prevent a decrease in classification accuracy due to the change. Achieve the results.
[0105]
The document classification method according to the present invention is a document classification method for classifying an electronic document into any one of preset categories. In the first feature vector space calculation step, a training document set is obtained. The feature vector space of each category is calculated from the feature vectors of each document constituting the feature vector, and in the second feature vector space calculation step, a feature vector space in which all the vectors have the same length, a feature vector space of the training document set And the feature vector space of each category calculated in the first feature vector space calculation step in the classification category calculation step. And the feature vector space of the unclassifiable category calculated in the second feature vector space calculating step, and classification By comparing the feature vector of each document that constitutes an elephant document set, one of the categories or the unclassifiable category is calculated as the category to which the document is classified, and the category addition necessity determination is performed. In the process, it is determined whether or not the frequency that the non-classifiable category is calculated as a classification destination category in the classification category calculation process exceeds a certain threshold value, and the classification cannot be classified in the category addition necessity determination process in the category addition recommendation process When it is determined that the category is a category to be classified and the frequency of calculation exceeds a certain threshold value, the operator is recommended to add a new category. Operators are encouraged to add new categories, so that they always maintain a classification system that matches the characteristics of the documents to be classified. Can, therefore an effect that document classification method capable of preventing a decrease in classification accuracy due to the change can be obtained.
[0106]
The document classification method according to the present invention is a document classification method for classifying an electronic document into any one of preset categories, and forms a training document set in a feature vector space calculation step. The feature vector space of each category is calculated from the feature vector of each document, and in the classification category calculation step, the feature vector space of each category calculated in the feature vector space calculation step and each document constituting the classification target document set By comparing with the feature vector, one of the categories is calculated as the category of the classification destination of each document. In the category deletion / merging necessity determination step, the classification category calculation step determines the classification destination It is determined whether there is a category whose category and calculated frequency are below a certain threshold. If it is determined that there is a category whose classification frequency and calculated frequency are below a certain threshold in the category deletion / merging necessity determination process, it is recommended that the operator delete or merge the category. When the number of documents classified into a certain category is reduced, it is recommended that the operator delete or merge the category, so that a classification system corresponding to the characteristics of the document group to be classified can always be maintained. There is an effect that a document classification method capable of preventing a reduction in classification accuracy due to the change can be obtained.
[0107]
Further, the document classification method according to the present invention is the above-described invention, further comprising a category division requirement for determining whether or not there is a category whose frequency calculated with the classification category in the classification category calculation step exceeds a certain threshold. When it is determined that there is a category in which the frequency calculated as a classification target category in the rejection determination step and the category division necessity determination step exceeds a certain threshold value, division of the category is recommended to the operator Category division recommending step, and when the number of documents classified into a certain category increases, the operator is recommended to divide the category, and accordingly, a classification system suitable for the characteristics of the classification target document group Therefore, there is an effect that a document classification method can be obtained that can always maintain the document quality, and can prevent a decrease in classification accuracy due to the change. That.
[0108]
Further, the recording medium according to the present invention records a program that causes a computer to execute the above method, so that the program can be read by the computer, and thereby the above method can be performed by the computer. There is an effect that a medium can be obtained.
[Brief description of the drawings]
FIG. 1 is an explanatory diagram showing a hardware configuration of a document classification device according to an embodiment of the present invention.
FIG. 2 is an explanatory diagram functionally showing the configuration of the document classification device according to the embodiment of the present invention.
FIG. 3 is an explanatory diagram for specifically explaining a method of creating a classification database 202c by a feature vector space estimation unit 202 in the document classification device according to the embodiment of the present invention;
FIG. 4 is an explanatory diagram showing an example of a classification target document set 201a and a feature vector created for each document in the document classification device according to the embodiment of the present invention;
FIG. 5 is an explanatory diagram showing an example of a classification result list 203a created by a classification category estimation unit 203 of the document classification device according to the embodiment of the present invention.
FIG. 6 is a flowchart showing a procedure of document classification processing of the document classification device according to the embodiment of the present invention.
FIG. 7 is an explanatory diagram showing an example of a classification result list window created by the user interface unit 205 of the document classification device according to the embodiment of the present invention.
FIG. 8 is a flowchart showing a procedure of category addition processing of the document classification device according to the embodiment of the present invention.
FIG. 9 is an explanatory diagram showing an example of a change recommendation dialog displayed by the user interface unit 205 of the document classification device according to the embodiment of the present invention.
FIG. 10 is an explanatory diagram showing an example of a category addition dialog displayed by the user interface unit 205 of the document classification device according to the embodiment of the present invention.
FIG. 11 is an explanatory diagram showing an example of a document assignment dialog displayed by the user interface unit 205 of the document classification device according to the embodiment of the present invention.
FIG. 12 is an explanatory diagram showing another example of a classification result list window displayed by the user interface unit 205 of the document classification device according to the embodiment of the present invention.
FIG. 13 is a flowchart showing a procedure of category division processing of the document classification device according to the embodiment of the present invention.
FIG. 14 is an explanatory diagram showing another example of the classification result list window displayed by the user interface unit 205 of the document classification device according to the embodiment of the present invention.
FIG. 15 is an explanatory diagram showing another example of the change recommendation dialog displayed by the user interface unit 205 of the document classification device according to the embodiment of the present invention.
FIG. 16 is an explanatory diagram showing an example of a category division dialog displayed by the user interface unit 205 of the document classification device according to the embodiment of the present invention.
FIG. 17 is an explanatory diagram showing another example of the document assignment dialog displayed by the user interface unit 205 of the document classification device according to the embodiment of the present invention.
FIG. 18 is an explanatory diagram showing another example of the classification result list window displayed by the user interface unit 205 of the document classification device according to the embodiment of the present invention.
FIG. 19 is a flowchart showing a procedure of category deletion / merging processing of the document classification device according to the embodiment of the present invention;
FIG. 20 is an explanatory diagram showing another example of a classification result list window displayed by the user interface unit 205 of the document classification device according to the embodiment of the present invention.
FIG. 21 is an explanatory diagram showing another example of the change recommendation dialog displayed by the user interface unit 205 of the document classification device according to the embodiment of the present invention.
FIG. 22 is an explanatory diagram showing an example of a category deletion / merging dialog displayed by the user interface unit 205 of the document classification device according to the embodiment of the present invention.
FIG. 23 is an explanatory diagram showing an example of a merge destination selection dialog displayed by the user interface unit 205 of the document classification device according to the embodiment of the present invention.
FIG. 24 is an explanatory diagram showing another example of the document assignment dialog displayed by the user interface unit 205 of the document classification device according to the embodiment of the present invention.
FIG. 25 is an explanatory diagram showing another example of the classification result list window displayed by the user interface unit 205 of the document classification device according to the embodiment of the present invention.
[Explanation of symbols]
100 buses
101 CPU
102 ROM
103 RAM
104 HDD
105 HD
106 FDD
107 FD
108 display
109 I / F
110 Communication line
111 keyboard
112 mouse
113 scanner
114 printer
115 CD-ROM
116 CD-ROM drive
201 Document storage unit
201a Classification target document set
202 Feature vector space estimation unit
202a Training document set
202b Document-category correspondence table
202c Classification database
203 Classification category estimation unit
203a Classification result table
204 Category System Management Department
204a Category system chart
204b Category system change rules
205 User interface

Claims (5)

In a document classification apparatus that classifies an electronic document into one of preset categories,
First feature vector space calculating means for calculating a feature vector space of each category from feature vectors of each document constituting a training document set;
A second feature vector space for calculating a feature vector space of an unclassifiable category from a weighted average of a feature vector space having the same length of all vectors, a feature vector space of a training document set, and a feature vector space of a classification target document set A calculation means;
The feature vector space of each category calculated by the first feature vector space calculating means and the feature vector space of the unclassifiable category calculated by the second feature vector space calculating means, and each of the components constituting the classification target document set Classification category calculation means for calculating any one of the respective categories or the unclassifiable categories as a classification destination category of each document by comparing with a feature vector of the document;
A category addition necessity judging means for judging whether or not the classification category calculating means determines whether the frequency of the classification impossible category and the category to be classified exceeds a certain threshold;
A category addition recommendation that recommends the operator to add a new category when it is determined by the category addition necessity determination means that the frequency that the non-classifiable category is calculated as a category to be classified exceeds a certain threshold. Means,
A document classification apparatus comprising: Further, a category division necessity determination unit that determines whether there is a category whose frequency calculated as a classification destination category by the classification category calculation unit exceeds a certain threshold value;
Category division recommendation means for recommending the division of the category to the operator when it is determined by the category division necessity determination means that there is a category whose frequency calculated as a classification destination category exceeds a certain threshold When,
The document classification apparatus according to claim 1 , further comprising: In a computer comprising a first feature vector space calculation means, a second feature vector space calculation means, a classification category calculation means, a category addition necessity determination means, and a category addition recommendation means, it is digitized A document classification method for classifying a document into one of preset categories,
A first feature vector space calculating step in which the first feature vector space calculating means calculates a feature vector space of each category from a feature vector of each document constituting a training document set;
The second feature vector space calculating means is characterized by a weighted average of a feature vector space in which all vectors are equal in length, a feature vector space of a training document set, and a feature vector space of a classification target document set. A second feature vector space calculating step for calculating a vector space;
The classification category calculation means includes a feature vector space of each category calculated in the first feature vector space calculation step, a feature vector space of an unclassifiable category calculated in the second feature vector space calculation step, and a classification A classification category calculation step of calculating any one of the categories or the unclassifiable categories as a classification destination category of the documents by comparing the feature vectors of the documents constituting the target document set;
A category addition necessity determining step, wherein the category addition necessity determining means determines whether or not the frequency that the unclassifiable category is calculated as the classification destination category in the classification category calculating step exceeds a certain threshold;
When the category addition recommendation means determines that the frequency of calculating the category that cannot be classified as the category to be classified in the category addition necessity determination step exceeds a certain threshold, the operator is prompted to add a new category. Recommended category addition process recommended for
Document classification method characterized by including Further, the computer includes a category division necessity determination unit and a category division recommendation unit,
The category division necessity determination means determines whether there is a category whose frequency calculated as a classification destination category in the classification category calculation step exceeds a certain threshold; and
When the category division recommendation means determines that there is a category whose frequency calculated as a classification destination category in the category division necessity determination step exceeds a certain threshold, the category division is performed for the operator. Recommended category division process,
The document classification method according to claim 3 , further comprising: In a document classification device that classifies an electronic document into one of preset categories using a computer,
The computer,
First feature vector space calculating means for calculating a feature vector space of each category from feature vectors of each document constituting a training document set;
A second feature vector space for calculating a feature vector space of an unclassifiable category from a weighted average of a feature vector space having the same length of all vectors, a feature vector space of a training document set, and a feature vector space of a classification target document set Calculation means,
The feature vector space of each category calculated by the first feature vector space calculating means and the feature vector space of the unclassifiable category calculated by the second feature vector space calculating means, and each of the components constituting the classification target document set Classification category calculation means for calculating any one of the respective categories or the unclassifiable categories as a classification destination category of each document by comparing with a feature vector of the document,
A category addition necessity judging means for judging whether or not the frequency at which the uncategorized category is calculated as a classification destination category by the classification category calculating means exceeds a certain threshold;
A category addition recommendation that recommends the operator to add a new category when it is determined by the category addition necessity determination means that the frequency that the non-classifiable category is calculated as a category to be classified exceeds a certain threshold. means,
A computer-readable recording medium characterized by recording a program for causing the functions as a.

Images