JP3264253B2 - Document automatic classification system and method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to an information classification device, and more particularly to an automatic document classification system.

[0002]

2. Description of the Related Art As an example of a conventional automatic classification system,
For example, Japanese Patent Application Laid-Open No. H10-111869 discloses a configuration of an apparatus for automatically classifying information such as text information or multimedia information based on various viewpoints for reusing document information and improving classification accuracy. Has been proposed. This conventional automatic classification system includes a teacher information feature amount extraction unit that extracts a feature of a category from a teacher information document given as an example in advance, and a new document information feature amount extraction unit that extracts a feature of a new document to be classified. And a new document classifying means for comparing the teacher information feature amount and the new document feature amount and classifying them into the most appropriate category.

The conventional automatic classification system having such a configuration operates roughly as follows. That is, teacher data is provided in advance to allow the system to learn the feature amount of the category, and the feature amount of each category is obtained based on this.

[0004] Next, a new document to be automatically classified is provided, and the characteristic amount of the document is obtained. Then, the document is compared with the characteristic amount of each category and classified into the most appropriate category.

[0005]

However, the above-mentioned conventional automatic classification system has a problem that it is not always appropriate to classify an actual document into one category when classifying an actual document. .

[0006] Specifically, for example, when classifying a variety of documents created by an unspecified number of people, it is often the case that documents whose contents span many categories are targeted.

[0007] If such a document is classified into any one of the categories, it becomes impossible to search from another category at the time of searching for reuse of the document.

Also, an algorithm that determines a classification into one category accurately requires a long time in its calculation process, and therefore, as described above, when an strict classification is not necessarily required, Is not efficient.

[0009] Therefore, the present invention has been made based on the recognition of the above technical problem, and an object of the present invention is to automatically classify documents into appropriate categories according to their contents, and to facilitate automatic search. It is to provide a classification system, apparatus and method.

[0010]

According to the present invention, there is provided an automatic classification system for classifying documents into a plurality of categories and for limiting the number of categories to be classified according to the contents of a genre. And classifying documents into a plurality of suitable categories according to their contents. More specifically, the present invention provides a data storage unit having a storage area divided into several sub-categories for a certain category, a feature learning unit for inputting learning data and learning in advance the characteristics of each of the sub-categories, Automatic classification means for analyzing characteristics of a document given as a classification target and determining a plurality of storage destination sub-categories from a semantic distance from the characteristics of each sub-category; Adjust the maximum number of storage destination subcategories and their combinations.

In the present invention, a plurality of different classifying means may be provided, and the classifying means may be switched according to the content of the category, so that the operation is efficient. More specifically, the automatic classification means comprises a plurality of automatic classification means of different classification schemes, means for selecting an appropriate automatic classification means according to the content of the category from the plurality of classification means, Further prepare. Alternatively, the automatic classification means has one classification means, and adjusts a classification method in accordance with the contents indicated by the category, based on data defining a given classification method.

[0012]

Embodiments of the present invention will now be described in detail with reference to the drawings. In a preferred embodiment, the automatic classification device of the present invention uses data storage means having storage areas divided into several sub-categories with respect to categories, and category learning data given in advance for category learning. Feature learning means for learning and extracting feature data for each category, analyzing a document to be classified based on the feature data for each category, and setting a plurality of sub-categories having similar contents to a range of a plurality of preset classification constraint conditions. Automatic classification means for classifying within.

The feature learning means merges documents belonging to the same category from the learning data given by the category learning data into one, and extracts the appearing words included in the merged document. Is calculated, and simple feature data of the subcategory is generated. Further, the automatic classification means extracts an appearance word of the given classification target document, calculates an appearance probability of each occurrence word, and generates simple feature data of the classification target document based on the occurrence probability of each occurrence word. Then, the simple feature data of the document to be classified is compared with the feature data of each sub-category extracted from the feature data of each category generated by the feature learning means, and a list of similar sub-categories is generated. From this list, It is configured to automatically classify into a plurality of sub-categories within a range of a predetermined multi-class constraint.

In a preferred second embodiment of the automatic classification apparatus of the present invention, the data storage means has a storage area divided into several sub-categories for each category, and each sub-category is recursively stored. A storage area having a hierarchical structure divided into several sub-categories, wherein the automatic classification unit uses the classification target document and the category of the document recursively passed from the data storage unit and the characteristic data for each category. Then, the classifying target document is classified into a plurality of subcategories having similar contents within the range of the plural classification constraint conditions.

In the third preferred embodiment, the automatic classification device of the present invention includes a storage area divided into several subcategories for each category, and each subcategory is recursively divided into several subcategories. A data storage means having a storage area having a divided hierarchical structure, and learning by using category learning data given in advance for category learning, and combining simple feature data and precise feature data of subcategories into each category. Feature learning means for generating feature data, first automatic classification means for classifying into a plurality of sub-categories having close contents using the feature data for each category within a range of a plurality of classification constraint conditions, and feature data for each category A second automatic classification means for accurately classifying into a single sub-category by using Depending on the document category that are passed from the recursively said data storage means and the target document, a switch means, for selecting one of the first and second automatic classification means.

In the fourth preferred embodiment, the automatic classification device of the present invention comprises a storage area divided into several subcategories for each category, and each subcategory is recursively divided into several subcategories. Data storage means having a storage area forming a hierarchical structure and learning by using category learning data given for the category learning in advance, and combining the simple feature data and the precise feature data of the sub-category with the feature for each category. A feature learning unit for generating data, a category of the document to be classified and a document category recursively passed from the data storage unit, and a description of a specification file defining a classification method based on the feature data for each category. Automatic classification means for selecting an appropriate classification method and performing automatic classification In the embodiment of the present invention, the features of the feature learning unit, the automatic classification unit, and the switch unit may be realized by a program executed on a computer. In this case, the present invention can be implemented by reading a program from a recording medium such as an FDD or a CD-ROM in which the program is recorded and executing the program. Hereinafter, the present invention will be described in detail with reference to examples.

[0017]

FIG. 1 is a diagram showing the configuration of a first embodiment of the present invention. Referring to FIG. 1, a first embodiment of the present invention relates to a data storage means 1 having a storage area divided into several sub-categories for a certain category, and a category learning provided to a system in advance for category learning. Based on feature learning means 2 for extracting feature data 7 for each category using data 5 and feature data 7 for each category,
Analyzing the classification target document 6 for performing automatic classification,
Automatic classification means 3 for classifying into a plurality of sub-categories having similar contents within the range of the plurality of classification constraint data 8.

Next, the operation of the first embodiment of the present invention will be described. First, it is necessary for the system to learn the characteristics of each subcategory before performing automatic classification.

FIG. 5 is a flowchart for explaining the processing operation of the feature learning means 2 in the first embodiment of the present invention.
The operation of the feature learning means 2 will be described with reference to FIGS.

First, the documents belonging to the same category among the learning data given by the category learning data 5 are merged into one (step S201).

Next, the words appearing in the merged document are cut out and extracted (step S20).
2). The appearance probability of each occurrence word is calculated (step S
203), and generate simple feature data of this sub-category (step S204). This is repeated until calculation is performed for all sub-categories (step S205).

The characteristic data 7 for each category obtained here
Automatically classifies a newly given document to be classified based on.

FIG. 6 is a flowchart for explaining the processing operation of the automatic classifying means 3 in the first embodiment of the present invention. The operation of the automatic classification means 3 will be described with reference to FIGS.

The feature data of all the sub-categories is extracted from the feature data 7 for each category (step S30).
1).

An appearance word of the newly given classification target document is cut out (step S302), and an appearance probability of each occurrence word is calculated (step S303).

Based on the appearance probabilities of the respective appearance words, simple feature data of the document is generated (step S30).
4).

The feature data generated in step S304 is compared with the feature data of each subcategory extracted in step S301 (step S305), and a list of similar subcategories is generated (step S3).
06).

From this list, automatic classification into a plurality of sub-categories is performed within the range of the predetermined plural-classification constraint data 8 (step S307).

The processing of steps S301 to S307 is performed for all the given new documents (step S308).

FIG. 12 is a diagram showing an example of the multiple classification constraint data 8 in the first embodiment of the present invention. Referring to FIG. 12, in the multiple classification constraint data 8, a category name 801, a maximum number of sub-categories 802 for which simultaneous classification is permitted for each category, and a combination 803 of sub-categories for which simultaneous classification is prohibited are defined for each category. I have. In the case of the example shown in FIG. 12, the maximum number of categories that can be classified simultaneously is “3” for the category of sports, and as a combination of subcategories that are prohibited from being classified at the same time as restrictions (Restriction), soccer and baseball, Combinations of soccer and golf, karate and baseball are specified.

Next, a second embodiment of the present invention will be described. FIG. 2 is a diagram showing the configuration of the second embodiment of the present invention. Referring to FIG. 2, a second embodiment of the present invention includes a storage area divided into several sub-categories for a certain category, and each sub-category is recursively divided into several sub-categories. And a data storage unit 10 having a storage area satisfying the following conditions. The feature learning unit 2 of the first embodiment, the document 6 to be classified, and the document recursively passed from the data storage unit 10 are classified by category. Automatic classification means 31 for classifying data into a plurality of subcategories having similar contents within the range of the plurality of classification constraint data 8 using the data 7.

The operation of the second embodiment of the present invention will be described. In the second embodiment of the present invention, as a feature learning method for each sub-category,
The same operation as the processing shown as a flowchart in FIG. 5 referred to in the description of the first embodiment is performed. Based on the characteristic data 7 for each category obtained here, a document to be newly given is automatically classified.

The data storage means 10 according to the second embodiment of the present invention has a hierarchical structure in which each category is recursively divided into sub-categories. When performing automatic classification, first perform the top-level sub-category classification,
Next, the documents included in the respective subcategories are recursively given to the automatic classification means 31 to automatically classify the documents into more detailed subcategories.

FIG. 7 is a flowchart for explaining the processing operation of the automatic classification means 31 in the second embodiment of the present invention. As shown in FIG. 7, the operation of the automatic classification means 31 in the second embodiment of the present invention is substantially the same as the processing operation of the automatic classification means 3 in the first embodiment described with reference to FIG. The sub-category to be classified is obtained from the category to which the document to be classified belongs at the beginning of the automatic classification operation, and the characteristic data is converted into the characteristic data for each category
6 (step S311 in FIG. 7) corresponds to FIG.
Is different from step S301 of FIG. In the flowchart of FIG. 7, steps S312 to S318 are performed in steps S302 to S308 in the flowchart of FIG.
Corresponding to

Next, a third embodiment of the present invention will be described. FIG. 3 is a diagram showing the configuration of the third exemplary embodiment of the present invention. Referring to FIG. 3, a third embodiment of the present invention includes the data storage means 10 of the second embodiment, and extracts feature data 71 for each category using category learning data 5 given in advance. Feature learning means 20
A first automatic classification means 31 for classifying into a plurality of sub-categories having similar contents within the range of the plural classification constraint data 8 using the characteristic data 7 for each category, and a single The second automatic classifying means 32 for accurately classifying the document into the subcategories, the classification target document 6 for performing the automatic classification, and the first and second categories according to the category of the document recursively passed from the data storage means 10.
And a switch 4 for selecting one of the automatic classification means 31 and 32.

Next, the operation of the third embodiment of the present invention will be described. First, it is necessary for the system to learn the characteristics of each subcategory before performing automatic classification.

FIG. 9 is a flow chart for explaining the processing operation of the feature learning means 20 according to the third embodiment of the present invention. The operation of the feature learning means 20 will be described with reference to FIGS.

First, the documents belonging to the same category among the learning data given by the category learning data 5 are merged into one (step S251).

Next, the words appearing in the merged document are cut out and extracted (step S25).
2). The appearance probability of each occurrence word is calculated (step S
253), and generate simple feature data of this sub-category (step S254).

Apart from this, precise feature data of this subcategory is also generated from the appearance probability of each word (step S255).

The two types of feature data are combined to generate feature data 71 for each category. This is repeated until calculation is performed for all subcategories (step S25)
6).

The characteristic data 7 for each category obtained here
1, a new document is automatically classified. In this system, according to a category to which a given document belongs, the switch 4 recursively receives data from the classification target document 6 and the data storage means 10 according to the first automatic classification means 31 for classifying the document into a plurality of categories, or , One of the second automatic classifying means 32 that classifies the data into a single category accurately.

As a criterion for this selection, the first automatic classification means 31 is selected in a category in which the contents represented by the sub-categories are likely to overlap, and the second automatic classification means 32 is selected in a category in which the contents represented by the sub-categories are difficult to be described simultaneously. It is set in the switch 4.

FIG. 8 is a flow chart for explaining the processing operation of the automatic classifying means 32 in the third embodiment of the present invention. The operation of the automatic classification means 32 will be described with reference to FIGS.

The precise feature data of the subcategory belonging to the category of the document to be categorized passed from the switch 4 is extracted from the feature data 71 for each category (step S321).

The appearance words of the given document are cut out (step S322), and the appearance probability of each appearance word is calculated (step S323). Based on this, precise document feature data is generated (step S32).
4).

The feature data generated in step S324 is compared with the feature data of the sub-category extracted in step S321 (step S325), and similar subcategories are listed (step S326).

Documents assigned to only the most similar subcategory from this list are automatically classified.
(Step S327).

The processes of steps S321 to S321 are performed on all the given new documents (step S328).

Next, a fourth embodiment of the present invention will be described. FIG. 4 is a diagram showing the configuration of the fourth embodiment of the present invention. Referring to FIG. 4, a fourth embodiment of the present invention is similar to the data storage means 1 described in the second embodiment.
0, and a feature learning unit 20 for extracting feature data 71 for each category using category learning data 5 given in advance, and a classification target document 6 and a document passed recursively from the data storage unit 10 are classified by a feature for each category. An automatic classification means 35 for selecting an appropriate classification method according to the description contents of the classification method definition file 8 based on the data 72 and automatically classifying the selected data.

FIG. 10 is a flowchart for explaining the processing operation of the automatic classification means 35 in the fourth embodiment of the present invention. FIG. 11 is a flowchart showing details of step 355 in FIG. The operation of the automatic classification means 35 will be described with reference to FIGS.

The precise feature data of the subcategory belonging to the category of the input document to be classified is extracted from the feature data 71 for each category (step S35).
1).

The appearance words of the given document are cut out (step S352), and the appearance probability of each appearance word is calculated (step S353). Based on this, precise document feature data is generated (step S35).
4).

Next, automatic classification is performed according to the classification method specified in the classification method specification file 81 given in advance (step S355). Referring to FIG. 11, step 35
The processing of step 5 is performed by combining the document characteristic data with the step S3.
The feature data of each sub-category extracted in 51 is compared (step S3551), and the data is classified into a plurality of similar categories (step S3553), and the data is classified into the single most similar category (step S355)
4) Select any one of the processes (step S3552) for determining a classification destination only based on the appearance of a specific word without comparing feature data.

Steps S351 to S35
Step 5 is performed for all the given documents (step S356).

[0056]

As described above, according to the present invention,
The following effects are obtained.

A first effect of the present invention is that a document including the contents of a plurality of categories can be appropriately classified.

The reason is that the present invention has a classification system that allows a plurality of classifications, and is configured to adjust the number of categories to be classified and the combination.

The second effect of the present invention is that, when there is a hierarchical category, classification can be performed efficiently.

The reason is that, in the present invention, in a category in which the contents of sub-categories are easily described in the same document at the same time, a plurality of classifications are performed at high speed by a simple algorithm.
For categories where the content of subcategories is difficult to describe in the same document, classify accurately with a precise algorithm,
This is due to such a configuration.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a first exemplary embodiment of the present invention.

FIG. 2 is a block diagram showing a configuration of a second exemplary embodiment of the present invention.

FIG. 3 is a block diagram illustrating a configuration of a third exemplary embodiment of the present invention.

FIG. 4 is a block diagram showing a configuration of a fourth embodiment of the present invention.

FIG. 5 is a flowchart for explaining the operation of the feature learning means 2 in the first and second embodiments of the present invention.

FIG. 6 shows an automatic classification unit 3 according to the first embodiment of the present invention.
3 is a flowchart for explaining the operation of FIG.

FIG. 7 is a flowchart for explaining the operation of the automatic classification means 31 in the second and third embodiments of the present invention.

FIG. 8 shows an automatic classification unit 3 according to a third embodiment of the present invention.
9 is a flowchart for explaining the operation of the second embodiment.

FIG. 9 is a flowchart for explaining the operation of the feature learning means 20 in the third and fourth embodiments of the present invention.

FIG. 10 is a flowchart for explaining the operation of the automatic classification means 35 according to the fourth embodiment of the present invention.

FIG. 11 is a flowchart for explaining the details of the operation of the automatic classification means 35 according to the fourth embodiment of the present invention.

FIG. 12 is a diagram illustrating an example of multiple classification constraint data 8 according to the first embodiment of this invention.

[Explanation of symbols]

 1, 10 Data storage means 2, 20 Feature learning means 3, 30, 31, 32, 35 Automatic classification means 4 Switch 5 Category learning data 6 Classification target document 7, 71 Feature data for each category 8 Multiple classification constraint data

Claims (14)

(57) [Claims] 1. A data storage means having a storage area divided into several sub-categories for a certain category, a feature learning means for inputting learning data and learning in advance the characteristics of each of the sub-categories, Automatically classifying means for analyzing the characteristics of the obtained document and determining a plurality of storage destination sub-categories from a semantic distance from the characteristics of the respective sub-categories, and storing destination sub-categories according to the content represented by the category of the document. Adjust the maximum number and combinations of
An automatic classification system characterized by being configured as follows. 2. The automatic classification system according to claim 1, wherein said data storage means has a storage area in which said sub-categories are further recursively divided into several sub-categories and form a hierarchical structure. 3. The automatic classifying means classifies a document to be classified into a plurality of sub-categories having similar contents by using a document to be classified and a category of the document recursively passed from the data storage means and characteristic data for each category. 3. The automatic classification system according to claim 2, wherein classification is performed within a range of a plurality of classification constraint conditions. 4. The automatic classification means further comprises: a plurality of classification means having different classification schemes; and a means for selecting an appropriate classification scheme from the plurality of classification means in accordance with the content of a category. 3. The automatic classification system according to claim 2, wherein: 5. The automatic classification means has one classification means, and further comprises means for adjusting a classification method in accordance with the contents indicated by the category based on data defining a given classification method. 3. The automatic classification system according to claim 2, wherein: 6. A data storage means having a storage area divided into several sub-categories with respect to a category, and learning using category learning data given in advance for category learning to extract feature data for each category. Automatic classification means for analyzing a classification target document based on the characteristic data for each category, and classifying the documents into a plurality of subcategories having similar contents within a range of a preset plural classification constraint condition. An automatic classification device, comprising: 7. The feature learning means for merging documents belonging to the same category among learning data given as category learning data into one, and extracting an appearing word included in the merged document. 7. The automatic classification apparatus according to claim 6, further comprising: means for calculating an appearance probability of each of the appearance words; and means for generating simple feature data of a subcategory. 8. An automatic classification means for extracting an appearance word of a given document to be classified and calculating an appearance probability of each of the appearance words; Means for generating simple feature data of the document; comparing the simple feature data of the classification target document with the feature data of each sub-category extracted from the feature data for each category generated by the feature learning means; 7. A means for generating a list of sub-categories to be performed, and means for automatically classifying the generated list into a plurality of sub-categories within a predetermined range of a plurality of classification constraint conditions. Automatic classification device as described. 9. The data storage means includes a storage area divided into several sub-categories for a category,
Further, each of the sub-categories has a storage area having a hierarchical structure recursively divided into a number of sub-categories, wherein the automatic classifying means includes a classification target document and a category of a document which is recursively passed from the data storage means. 7. The automatic classification device according to claim 6, wherein the classification target document is classified into a plurality of subcategories having similar contents within a range of a plurality of classification constraint conditions using the feature data for each category. . 10. A data storage means having a storage area divided into a number of sub-categories with respect to a category, wherein category learning data is input in advance to learn features of each of the sub-categories, and feature data for each category is created. When automatically classifying the document to be classified, the feature data of the sub-category is extracted from the feature data for each category, feature data is generated for the document to be classified, and the generated feature data and each of the extracted sub-categories are extracted. , And a list of similar subcategories is listed. From this list,
An automatic classification method characterized by automatically classifying into a plurality of sub-categories within a predetermined range of constraint conditions of a plurality of classification data. 11. A data storage means comprising a storage area divided into several sub-categories for each category, and further comprising a storage area having a hierarchical structure in which each sub-category is recursively divided into several sub-categories, The category learning data is input in advance to learn the characteristics of each of the sub-categories to create feature data for each category. When automatically classifying the classification target documents, the classification target documents and the data storage means Using the category of the document passed from and the feature data for each category, the classification target document,
An automatic classification method characterized by classifying into a plurality of subcategories having similar contents within a range of a plurality of classification constraint conditions. 12. A sub-category to be classified is determined from the category to which the document to be classified belongs, and the characteristic data is extracted from the characteristic data for each category, the top sub-category is classified, and then the sub-category is classified. the including document by recursively automatic classification and automatic classification into more detailed sub-categories, automatic classification method of claim 1 1, wherein a. 13. A data storage means having a storage area divided into a number of sub-categories with respect to a certain category. (A) Category learning data is input in advance to learn the characteristics of each of the sub-categories, and (B) When automatically classifying the documents to be classified,
The feature data of the sub-category is extracted from the feature data of each category, the feature data is generated for the classification target document, the generated feature data is compared with the feature data of each of the extracted sub-categories, A list is listed up, and from the list, processing of automatically classifying into a plurality of sub-categories within the range of the constraint condition of a plurality of classification data given in advance is executed on a computer by the above-described processing of (a) and (b). A recording medium on which a program for causing a program to be recorded is recorded. 14. A data storage means comprising a storage area divided into several sub-categories with respect to a category, and further comprising a storage area having a hierarchical structure in which each sub-category is recursively divided into several sub-categories, (A) a process of inputting category learning data in advance and learning features of each of the sub-categories to create feature data for each category; and (b) a process of retrieving documents to be classified and documents recursively passed from the data storage means. Using the category and the feature data for each category, the document to be classified is
A recording medium for recording a program for causing a computer to execute the above-described processes (a) and (b) of a process of classifying into a plurality of subcategories having similar contents within a range of a plurality of classification constraint conditions.