JP2004054303A - System for making electronic dictionary for document classification and system using it for classifying document - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve a certainty of application of a proper category to a document to be classified. <P>SOLUTION: A system dictionary 13 or user dictionaries 11A, 11B... listing respective character strings and one or more categories serving as mapping of the character strings are prepared. A text classifying device 1 acquires a classification objective text 19 and decides a candidate of the category to be applied to the obtained classification objective text 19 on the basis of the system dictionary 13, the user dictionaries 11A, 11B.... The text classifying device 1 displays a user interface and the like for receiving a cause why the category is nominated for application, an inquiry asking whether the category may be applied or not, and an answer to the inquiry. When an affirmative answer is received to its inquiry, the text classifying device 1 applies the category to the classification objective text 19. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a novel technique for creating an electronic dictionary for document classification based on a plurality of sample documents, and a novel document classification technique using the electronic dictionary.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, there has been known a text classification system for automatically classifying a document including only characters (hereinafter, referred to as “text”). In a conventional text classification system, automatic text classification is performed as follows.
[0003]
That is, in the conventional text classification system, for each of a plurality of text categories, a learning text group including a large number of learning texts (sample texts) to which the categories are assigned is prepared. The text classification system creates word statistical data (for example, data indicating the distribution of each word in the category) by extracting each word from the learning text group and performing statistical processing for each category. Then, when a text to be classified (hereinafter referred to as a text to be classified) is input, the text classification system extracts each word from the input text to be classified, and generates each extracted word and each word. By assigning a category to the classification target text based on the word statistical data, the classification target text is automatically classified.
[0004]
[Problems to be solved by the invention]
The conventional text classification system has the following problems, for example.
[0005]
That is, since the classification method is based on statistical data, for example, (1) a problem that a huge amount of calculation is required for dictionary construction and classification determination, and (2) statistical data is merely a numerical value. (3) that humans cannot obtain information from the system to determine whether the classification result is correct, and (4) that the content of the text to be classified is accurate. Therefore, there is a problem that a person who does not know the contents of the text to be classified cannot judge the appropriateness of the assigned category at all.
[0006]
Conventionally, since the above-described automatic classification is merely performed based on statistical data created for each category, it is necessary to feed back a category finally assigned to a classification target text by a human to reconstruct a dictionary. Since it requires a huge amount of processing time, it is practically difficult.
[0007]
It is considered that the above problems exist not only in text but also in other types of documents including images and line drawings.
[0008]
Therefore, an object of the present invention is to improve the certainty that an appropriate category is assigned to a document to be classified.
[0009]
Another object of the present invention is to make it possible to easily determine whether a category assigned to a document to be classified is appropriate.
[0010]
Still another object of the present invention is to enable appropriate feedback to be provided to a system for automatically classifying documents based on the result of a category determined by a human for a document to be classified. is there.
[0011]
[Means for Solving the Problems]
A system according to a first aspect of the present invention includes an association degree determination unit and a dictionary creation unit. The relevance determination means is a system for creating an electronic dictionary for classifying documents, and inputs a plurality of sample documents belonging to each of a plurality of categories of the document, and, for each category, For each of a large number of character strings included in the document, the degree of association between the character string and the category is determined. The dictionary creating means selects, based on the result of the determination, each specific character string whose degree of association is equal to or more than a predetermined degree from among the many character strings, and selects each of the selected specific character strings and the corresponding specific character string. An electronic dictionary in which the obtained one or more categories are recorded is created.
[0012]
Here, the “document” is a document including characters, for example, a document including only characters, or a document including an image or a line drawing in addition to characters.
[0013]
The "category" may be anything as long as it makes the document categorizable, for example, words that can be understood by humans (for example, novels, essays, etc.), or a code group consisting of numbers, alphabetic characters, and the like. It may be.
[0014]
The “character string” is, for example, a grammatical word (that is, a minimum unit of language), a continuous word (for example, a character string “document creation system”), or a plurality of words (for example, Or a character string consisting of multiple particles and multiple words.
[0015]
In a preferred embodiment of the system according to the first aspect of the present invention, the relevance determination means is included in (1) the plurality of sample documents belonging to the same category for each of the plurality of character strings. Based on the total number of the character strings and (2) the total number of the existing sample documents or the total number of the character strings included in the existing sample documents, the degree of association with respect to the character strings is determined.
[0016]
A system according to a second aspect of the present invention is a system for classifying documents, and includes a dictionary acquisition unit, a document acquisition unit, a comparison / collation unit, and a document classification unit. The dictionary acquisition means acquires an electronic dictionary in which one or more character strings corresponding to each of a plurality of categories of the document (for example, character strings characteristic to the category) are described. The document input unit acquires a document to be classified (for example, receives a document to be classified from a user, or creates a document according to a user's request to acquire a document to be classified). The comparison and collation unit compares and compares each character string included in the acquired document to be classified with each character string (for example, the characteristic character string described above) described in the electronic dictionary. Do. As a result of the comparison and collation, if there is a character string that matches each other, the document classifying unit refers to the electronic dictionary to grasp a category corresponding to the matched character string, and The acquired document is classified by assigning the category to be assigned, which is the above-identified category.
[0017]
In a preferred embodiment of the system according to the second aspect of the present invention, the system further includes a basis character string extracting unit, a notifying unit, a question output unit, and an answer receiving unit. The basis character string extracting means extracts, from the acquired document, the matched character string or the first long character string including the matched character string, which became the basis for the provision of the category to be provided. The notification unit notifies the user who has input the document of the extracted matched character string or the first long character string and the assignment target category. The question output means outputs to the user a question as to whether the notified category to be provided may be assigned to the acquired document. The answer receiving means receives an answer to the output question from the user.
[0018]
In this case, the document classifying means assigns the assignment target category to the acquired document only when the answer receiving means receives a positive answer.
[0019]
In a further preferred embodiment of the present invention, the basis character string extracting means includes the plurality of sample documents belonging to each of the plurality of categories, the specific sample document including the matched character string, A second long character string including the matched character string is extracted from the specific sample document, and the notifying unit determines the first long character string, the assignment target category, and the second long character string. And the category to which the specific sample document belongs to the user (for example, the combination of the first long character string and the category to be given, the second long character string and the The combination with the category to which the sample document belongs is displayed side by side on a predetermined screen).
[0020]
In a preferred embodiment, there are a plurality of electronic dictionaries, and the plurality of electronic dictionaries include a user dictionary created by each user and dedicated to each user. Further comprising dictionary editing means for editing a user dictionary dedicated to the user in response to a request from the user.
[0021]
A method according to a third aspect of the present invention is a method for creating an electronic dictionary for classifying documents, comprising inputting a plurality of sample documents belonging to each of a plurality of categories of documents, and For each of a number of character strings included in the plurality of sample documents, a step of determining the degree of association between the character string and its category; and, based on the result of the determination, Selecting, from among them, specific character strings whose degree of association is equal to or greater than a predetermined degree, and creating an electronic dictionary in which the selected specific character strings and one or more categories corresponding thereto are recorded. Having.
[0022]
A method according to a fourth aspect of the present invention is a method for classifying a document, wherein a character string corresponding to each of a plurality of categories of the document and a character string characteristic to the category is 2 or more. Obtaining an electronic dictionary in which at least one document is described, obtaining a document to be classified, and each character string included in the obtained document, and being described in the electronic dictionary. Performing a comparison and collation with each characteristic character string, and if there is a character string that matches each other as a result of the comparison and collation, refer to the electronic dictionary to find a category corresponding to the matched character string. And classifying the acquired document by assigning an assignment target category that is the identified category to the acquired document.
[0023]
A computer program according to a fifth aspect of the present invention is a computer program for creating an electronic dictionary for document classification, and inputs a plurality of sample documents belonging to each of a plurality of categories of documents, and For each category, for each of a large number of character strings included in the plurality of sample documents, determining a degree of association between the character string and the category; and, based on a result of the determination, determining the number of the character strings. From the columns, select each specific character string whose degree of association is equal to or higher than a predetermined degree, and create an electronic dictionary in which the selected specific character string and one or more categories corresponding to the selected specific character string are recorded. And a step for causing a computer to execute the steps.
[0024]
A computer program according to a sixth aspect of the present invention is a computer program for classifying a document, and includes a character string corresponding to each of a plurality of categories of the document (for example, for the category, Obtaining an electronic dictionary (in which two or more characteristic character strings are described), obtaining a document to be classified, each character string included in the obtained document, Performing a comparison and comparison with each characteristic character string described in the electronic dictionary; and as a result of the comparison and collation, if there is a character string that matches each other, The category corresponding to the matched character string is grasped, and the acquired document is classified by assigning the granted category which is the grasped category to the acquired document. That it is intended to execute the steps in the computer.
[0025]
The data according to the seventh aspect of the present invention is an electronic dictionary for document classification, which corresponds to each of a plurality of categories of a document, and is considered or characteristic of the category. Contains one or more character strings. One or more character strings are associated with one category. For example, the same category may be associated with two types of character strings, or two types of categories may be associated with the same character string. In the former case, no matter which of the two types of character strings appears, the same category can be assigned. In the latter case, if the same character string appears, the two categories must be assigned. Both can be provided.
[0026]
The plurality of means provided in each system of the present invention described above can be mounted on one computer system, or can be mounted separately on a plurality of distributed computer systems.
[0027]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[0028]
FIG. 1 shows an overall configuration of a system according to an embodiment of the present invention.
[0029]
In this embodiment, a learning text group storage unit 3, a dictionary creation device 5, and a text classification device 1 are provided. Note that the dictionary creation device 5 and the text classification device 1 may each be one computer machine and be physically independent, or each may be one application program and one computer machine. Or may be mounted separately on two computer machines.
[0030]
The learning text group storage unit 3 stores a plurality of learning text groups 21A, 21B,..., For example, as shown in FIG. Each learning text group, for example, the learning text group 21A is composed of a plurality of learning texts 21A to which the same category (for example, “category A”) of the text is assigned. ₁ , 21A ₂ , 21A ₃ (The same applies to the other learning text groups 21B ...). As a result, each learning text group is associated with one of a plurality of categories. As described above, the “learning text” is a text to which a category has already been assigned and is a sample text (for example, a text that has been classified in the past). When two or more categories are assigned, the learning texts are present in each learning text group corresponding to the two or more categories.
[0031]
The dictionary creation device 5 extracts each character string from each learning text group stored in the learning text group storage unit 3 and calculates the importance of each extracted character string in the category corresponding to the extracted character string. An electronic dictionary for text classification is created on the basis of the result of the assigned importance. The created dictionary is output as a system dictionary 13 described later (for this reason, the electronic dictionary is hereinafter referred to as a “system dictionary”).
[0032]
The text classification device 1 can be used by a plurality of users. The text classification device 1 acquires a text to be classified (hereinafter referred to as a text to be classified) 19 by a specific method, and a system dictionary 13 created by the dictionary creation device 5 and an electronic dictionary created by a plurality of users. (Hereinafter, user dictionary) 11A, 11B,..., And at least one of the plurality of categories described above is assigned to the acquired classification target text 19. The text classification device 1 includes a system dictionary storage unit 7, a user dictionary storage unit 9, a text classification unit 15, and a user dictionary update unit 17.
[0033]
The system dictionary storage unit 7 stores the system dictionary 13 created and output by the dictionary creating device 5 (the stored system dictionary 13 is hashed, for example). For example, as shown in FIG. 3, the system dictionary 13 describes each characteristic character string (indicated as “phrase” in FIG. 3) and one or more categories corresponding to each characteristic character string. ("Characteristic character string" is a character string estimated to be characteristic in the category by the dictionary creation device 5 in a method described later). The system dictionary 13 is stored in a specific manner (for example, via a portable recording medium such as a floppy (registered trademark) disk or MO disk, or using a wired or wireless communication system). Can be taken into the system dictionary storage unit 7 of the text classification device 1.
[0034]
As shown in FIG. 4, for example, the user dictionary storage unit 9 stores user dictionaries created by a plurality of users (specifically, for example, the user dictionary 11A of the user A, the user dictionary 11B of the user B,. .) Are stored (the stored user dictionaries 11A, 11B,... Are, for example, hashed). In each user dictionary, for example, the user dictionary 11A, one or more desired character strings of the user A (indicated as “words” in FIG. 4, hereinafter referred to as “user desired character strings”), and one corresponding to each user desired character string. Or, two or more categories are described (the configuration of the other user dictionaries 11B,... Is the same). Each of the user dictionaries 11A, 11B,... Can be created by using a predetermined application program installed in the text classification device 1, or can be an external device (not shown) (for example, a personal computer or a user output therefrom). It can also be stored in the user dictionary storage unit 9 by inputting the text classification device 1 from a portable storage medium storing a dictionary).
[0035]
The text classification unit 15 acquires the classification target text 19 by a specific method, and the system dictionary 13 stored in the system dictionary storage unit 7 and the user dictionaries 11A, 11B,. Based on the above, a category to be assigned to the acquired classification target text 19 (hereinafter, an assignment target category) is grasped from the plurality of categories described above. Then, the text classifying unit 15 displays a question as to whether or not the grasped grant target category can be assigned to the sort target text 19 (hereinafter, a question of whether the assignment is appropriate). When the text classifying unit 15 receives an affirmative answer to the assignment eligibility question, the text classification unit 15 assigns the above-described assignment target category to the classification target text 19, and receives a negative answer to the assignment eligibility question. In this case, a predetermined process (for example, a process of searching for another category that can be an application target) is executed.
[0036]
The user dictionary updating unit 17 receives a request from a certain user among a plurality of users, and responds to the request of the user by using one or more user dictionaries 11A and 11B stored in the user dictionary storage unit 9. , ..., the user dictionary corresponding to the user is edited and updated.
[0037]
The above is the outline of the system according to the present embodiment. Hereinafter, the processing flow in this system, in other words, the processing contents performed by the dictionary creation device 5, the text classification unit 15, and the user dictionary update unit 17 will be described in detail.
[0038]
FIG. 5 shows a processing flow when the dictionary creation device 5 creates the system dictionary 13. In addition, in FIG. 5 and subsequent drawings and the following description, “step” is abbreviated as “S”.
[0039]
The dictionary creation device 5 is activated when a predetermined event occurs (for example, when a request is received from a user). First, the dictionary creation device 5 extracts each character string from each learning text included in each learning text group 21A, 21B,... In the learning text group storage unit 3 (S1). Here, the “character string” may be a grammatical word (that is, a minimum unit of language) or a continuous word (for example, as shown in FIG. 3, “automatic focus adjustment”). ), Or a plurality of words connected via a particle (for example, it may be "mine"), or a plurality of particles and a plurality of words. It may be a character string (for example, "I filed a patent application").
[0040]
Next, the dictionary creation device 5 determines, for each of the extracted character strings, the importance in the category (hereinafter, referred to as “corresponding category”) to which the learning text that includes the character string belongs, in other words, The degree of association between each extracted character string and its corresponding category is calculated. Specifically, for example, for each extracted character string, the dictionary creation device 5 calculates the appearance ratio of the character string that appears in the corresponding category with respect to the entire character string (hereinafter, referred to as “character string appearance ratio”). (S2). The dictionary creation device 5 uses the following equation (1)
Character string appearance ratio = appearance frequency in learning text group corresponding to corresponding category / appearance frequency in all learning text groups (1)
Is used to calculate the character string appearance ratio for each of the extracted character strings. Here, the “appearance frequency” refers to the number of appearances (the above equation (1) is an example, and although not specifically described, there are other calculation methods).
[0041]
The dictionary creation device 5 determines whether the calculated character string appearance ratio exceeds a predetermined threshold value for each of the extracted character strings. As a result of the determination, the dictionary creation device 5 sets a character string for which a positive result is obtained (that is, a character string whose character string appearance ratio exceeds a predetermined threshold) as a candidate to be registered in the system dictionary ( S3) (hereinafter, the character string is referred to as “registration candidate character string”).
[0042]
Here, among the character string appearance ratios of a plurality of registration candidate character strings, the parameter (that is, the value of the denominator in the above equation (1)) was not more than a predetermined value when calculating the character string appearance ratio. There is a possibility that there is a character string appearance ratio that exceeds the threshold. Therefore, the dictionary creation device 5 performs a population ratio test on each registered candidate character string whose registered parameter is equal to or less than a predetermined number when calculating the character string appearance ratio thereof. Then, it is determined whether it is appropriate to register the registration candidate character string in the system dictionary 13 (S4).
[0043]
Thereafter, the dictionary creating device 5 determines whether the registration candidate character string determined to be valid in the determination in S4 and the registration candidate character string not determined in S4 (that is, there is no need to perform the determination in S4). The registered candidate character string) and the corresponding category are recorded in a predetermined file serving as the system dictionary 13 (S5).
[0044]
By completing the above processing flow, only a character string whose importance is equal to or more than a certain value in each category (in other words, a character string estimated to be characteristic in each category) and a corresponding category A recorded system dictionary 13 is created. Based on a plurality of character strings described in the system dictionary 13 (hereinafter referred to as “characteristic character strings”) and a plurality of user desired character strings described in the user dictionaries 11A, 11B,. Then, the text classification unit 15 classifies the classification target text 19.
[0045]
FIG. 6 shows a processing flow when the text classification unit 15 classifies the classification target text 19.
[0046]
The text to be classified 19 is input to the text classification device 1 by a user using a specific method (S10). Here, the “specific method” means, for example, that a new text to be classified is created in the text classification device 1 in response to a user request, or a portable recording medium such as a floppy (registered trademark) disk. Or via a wired or wireless communication system.
[0047]
When the classification target text 19 is input, the text classification unit 15 extracts each character string described in the classification target text 19 (S11).
[0048]
Then, the text classification unit 15 accesses the user dictionary storage unit 9, refers to the user dictionary 11A of the user who has input the classification target text 19 (hereinafter, the user is referred to as “user A”), and extracts the text in S11. Of the extracted character strings (hereinafter, the character strings are referred to as “extracted character strings”) and one or more user-desired character strings described in the user dictionary 11A (hereinafter, “user dictionary comparison and collation”). Is executed (S12). Note that which of the plurality of user dictionaries 11A, 11B,... Is the user dictionary of the user A can be determined by a predetermined method (for example, a user identification code is assigned to each of the user dictionaries 11A, 11B,. If the user A enters the user identification code of the user A, the input user identification code is compared with the user identification codes associated with the user dictionaries 11A, 11B,. Can be used to determine the user dictionary 11A). In S12, for example, the text classification unit 15 shares a dictionary with a group of a plurality of users, or creates a group dictionary that can be used by the users of the group, and creates at least one of the shared dictionary and the group dictionary. The above-described user dictionary comparison / collation may be executed using the above.
[0049]
After S12, the text classification unit 15 accesses the system dictionary storage unit 7 and refers to the system dictionary 13, and refers to each character string extracted in S11 and one or more user-desired characters described in the system dictionary 13. A comparison and collation with a column (hereinafter, referred to as “system dictionary comparison and collation”) is executed (S13).
[0050]
The text classifying unit 15 preferentially uses the result of the user dictionary comparison and collation of the result of the user dictionary comparison and collation of S12 and the result of the system dictionary comparison and collation of S13, and determines the category to be assigned to the classification target text 19. Obtain a candidate (Note that the user dictionary comparison / matching result does not necessarily have to be used preferentially. The system dictionary comparison / matching result may be used preferentially, or both results may be compared without any difference. May be used).
[0051]
Specifically, for example, as a result of the user dictionary comparison and collation in S12, if the text classification unit 15 determines that there is an extracted character string that matches the user desired character string on the user dictionary 11A among a plurality of extracted character strings, With reference to the user dictionary 11A, a category associated with the user-desired character string that matches the extracted character string is identified, and the category is set as a candidate to be assigned to the classification target text 19 (this processing is performed). May be performed after execution of S12 and before execution of S13).
[0052]
In addition, as a result of the user dictionary comparison and collation in S12, the text classification unit 15 determines that there is no extracted character string that matches the user desired character string on the user dictionary 11A among the plurality of extracted character strings, and the system dictionary in S13. As a result of the comparison and collation, if there is an extracted character string that matches the characteristic character string in the system dictionary 13 among the plurality of extracted character strings, the feature that matches the extracted character string is referred to by referring to the system dictionary 13. The category associated with the target character string is identified, and the category is determined as a candidate to be assigned to the classification target text 19.
[0053]
When the category that is a candidate to be assigned (hereinafter, referred to as “assignment candidate category”) is determined by the above processing, the text classifying unit 15 assigns the assignment candidate category and the assignment candidate category to the classification target text 19. A process is executed for displaying a question of whether or not to allow the display on a predetermined display screen (not shown) (S14).
[0054]
Specifically, for example, when the text classifying unit 15 determines one or more assignment candidate categories based only on the result of the user dictionary comparison / collation in S12, the reason that each assignment candidate category is an assignment candidate A long character string (for example, a character string including 50 characters before and after the character string of the ground) including the character string (hereinafter, referred to as “ground character string”) is extracted from the classification target text 19 and extracted. A process is executed for displaying a long character string, each assignment candidate category, an assignment suitability question, and a user interface for receiving an answer to the assignment suitability question.
[0055]
For example, when the text classification unit 15 determines one or more assignment candidate categories (for example, a category “category G”) based on the result of the system dictionary comparison / collation in S13, as illustrated in FIG. The long character string 70 including the base character string (“for electrostatic photography” in the example of FIG. 7) for the assignment candidate category “category G” is extracted from the classification target text 19. Further, the text classifying unit 15 accesses the learning text group storage unit 3 and performs the learning including the ground character string “for electrostatic photography” in the learning text group corresponding to the assignment candidate category “category G”. Textbook 21G ₁ , 21G ₃ Identify. In addition, the text classifying unit 15 determines each of the learning texts 21G. ₁ , 21G ₃ , The long character strings 50 and 60 including the ground character string “for electrostatic photography” are extracted. Then, the text classifying unit 15 executes a process for generating the assignment propriety response acceptance screen 100 based on the extracted long character strings 50, 60, and 70 and displaying the same (of course, such as The processing may be performed when one or more assignment candidate categories are determined based only on the result of the user dictionary comparison and collation in S12 and there is a learning text belonging to the assignment candidate category.)
[0056]
The assignment appropriateness response reception screen 100 includes a long character string 70 extracted from the classification target text 19 and each learning text 21G. ₁ , 21G ₃ Are displayed side by side on the left and right. In addition, each of the learning texts 21G ₁ , 21G ₃ Are displayed, and the category "Category G" corresponding to the present application is also displayed. Further, on the grant aptitude response reception screen 100, the grant aptitude question 110, an “OK” button 111 operated when giving an affirmative answer to the grant aptitude question 110, and a negative A "change" button 113 that is operated when making an answer and a "update user dictionary" button 115 that is operated when the user wants to edit the user dictionary 11A are displayed.
[0057]
With this screen configuration, the user A can determine the cause 70 of “category G” as a candidate for the classification target text 19 and the existing learning text 21G. ₁ , 21G ₃ Can be compared with the causes 50 and 60 to which "category G" is assigned, so that it is easy to determine whether or not it is appropriate to assign "category G" to the classification target text 19. it can.
[0058]
The user A operates the “OK” button 111 when looking at the acceptability answer reception screen 100 and determining that “category G” is appropriate to be assigned to the classification target text 19. When the “OK” button 111 is operated (Y in S15 of FIG. 6), the text classification unit 15 assigns “category G” to the classification target text 19 (S17), thereby classifying the classification target text 19. Finish. At this time, the text classification unit 15 feeds back to the user dictionary 11A or the system dictionary 13 that "category G" has been assigned to the classification target text 19 including the character string "for electrostatic photography". Is also good.
[0059]
The user A operates the “change” button 113 if he sees the assignment aptitude response reception screen 100 and determines that it is not appropriate for “category G” to be assigned to the classification target text 19. When the “change” button 113 is operated (N in S15 of FIG. 6), the text classification unit 15 executes a predetermined operation, for example, a process of searching for another assignment candidate category (S16). At this time, the text classification unit 15 does not assign “category G” to the user dictionary 11A or the system dictionary 13 even though the classification target text 19 includes the character string “for electrostatic photography”. You may give feedback to the effect.
[0060]
The user A operates the “update user dictionary” button 115 if he / she wants to edit the user dictionary 11A while viewing the acceptability answer reception screen 100. In this case, the user dictionary updating unit (see FIG. 1) 17 is started in response to the operation, and thereafter, only the user dictionary 11A is edited and updated according to the request of the user A (user dictionary 11B of another person). Is not editable). That is, based on the result of the classification of the classification target text 19, the user A gives feedback to his or her own user dictionary 11A.
[0061]
In the flow shown in FIG. 6, as a result of the processing in S12 and S13, none of the plurality of character strings included in the classification target text 19 must be described in the user dictionary 11A or the system dictionary 13 at all. For example, no classification candidate category is obtained for the classification target text 19. In this case, the text classification unit 15 executes a process for displaying that the classification target text 19 could not be classified. Thereafter, when there is a request from the user A to the text classifying device 1 that the user A itself classifies the text to be classified 19, the following processing is performed.
[0062]
That is, as shown in FIG. 8, the classification target text 19 'to which no category is assigned is displayed, and in response to the operation of the user A, the character string desired by the user in the classification target text 19' (FIG. In the example, "inkjet printer" is identified (for example, a character string shaded by the user A operating the mouse is identified). After that, when the user A inputs to the text classification device 1 that "category B" is to be assigned to the classification target text 19 ', the user dictionary update unit 17 stores the user dictionary A in the user dictionary 11A of the user A. As a new record, a character string “inkjet printer” desired by the user A and “category B” as a category corresponding thereto are registered. Note that the number of combinations of character strings and categories that can be registered in the user dictionary 11A is not limited to one. Further, the number of categories associated with one character string is not limited to one.
[0063]
The above is the description of the present embodiment. The above-described embodiment can be used for classifying other types of documents including images as well as texts, and can also be used for classifying documents for various purposes. Specifically, for example, in the above embodiment, the learning text is a patent specification to which an international patent classification has already been assigned, the category is an international patent classification, and the classification target text is not assigned an international patent classification. In the case of a patent specification, an international patent classification is automatically assigned to the patent specification.
[0064]
As described above, according to the above-described embodiment, a category is assigned to the classification target text 19 using the system dictionary 13 and the user dictionaries 11A, 11B,. The system dictionary 13 and the user dictionaries 11A, 11B,... Are different from mere statistical data, and describe each character string and one or more categories that map the character string. Therefore, if the classification target text 19 includes the same character strings as those on the system dictionary 13 or the user dictionaries 11A, 11B,..., The appropriate category is automatically classified with a higher probability than before. The text 19 is given.
[0065]
Further, according to the above-described embodiment, the assignment appropriateness response reception screen 100 illustrated in FIG. 7 is generated and displayed. It is easy to determine whether it is appropriate to assign a certain category to the classification target text 19 by comparing causes 50 and 60 to which a certain category is assigned to an existing learning text. can do. As a result, it is possible to increase the certainty that the category given to the classification target text 19 is appropriate, and it is easy to construct a system on the user side.
[0066]
Further, according to the above-described embodiment, the user can appropriately edit his / her own user dictionary. This is also effective as a support environment for customizing the user dictionary.
[0067]
The preferred embodiment of the present invention has been described above, but this is an exemplification for describing the present invention, and is not intended to limit the scope of the present invention only to this embodiment. The present invention can be implemented in other various forms.
[0068]
For example, the content of the learning text group storage unit 13 is updated periodically or irregularly, and the dictionary creation device 5 executes the processing flow shown in FIG. It may be updated on a regular basis or irregularly.
[0069]
Further, for example, the dictionary creation device 5 determines a predetermined area from a plurality of areas on the learning text (for example, in the case of a patent specification, for example, a character string “name of invention” and “ It is also possible to determine the area between the character string "claim range"), extract the character string only from the determined area, and use the extracted character string as a characteristic character string.
[0070]
In addition, for example, the user prepares one or more sample texts by himself (hereinafter, the sample text prepared by the user is referred to as “user text”), and the user text (for example, a text created by the user). The texts referred to when updating the user dictionary, such as a text document or a Web page downloaded from a desired site, may be stored in the learning text group storage unit 3 for each category. Accordingly, when the classification target text 19 is classified by category, not only a learning text prepared in advance but also a user text prepared by a user can be used as a word (hereinafter referred to as a basis) for determining a classification candidate category. , "Ground word") is extracted, and the character string and the character string containing the ground word on the classification target text 19 are displayed on the acceptance / rejection answer acceptance screen 100. Displayed side by side. Therefore, it is possible to more easily and accurately determine the category assigned to the classification target text 19. At this time, both or one of the learning text and the user text may be used, or the user sets in advance the priority of which text is to be used preferentially, and One of the learning text and the user text may be automatically and selectively used based on the. In the latter case, for example, if the ground word does not exist on one of the previously used texts (for example, the user text), the ground word is automatically searched from the other text (for example, the learning text). May be.
[Brief description of the drawings]
FIG. 1 is a block diagram showing an overall configuration of a system according to an embodiment of the present invention.
FIG. 2 is a diagram showing a state in a learning text group storage unit 3;
FIG. 3 is a diagram showing a configuration of a system dictionary 13;
FIG. 4 is a diagram showing a configuration of user dictionaries 11A, 11B,.
FIG. 5 is a diagram showing a processing flow when the dictionary creation device 5 creates a system dictionary 13;
FIG. 6 is a diagram showing a processing flow when the text classification unit 15 classifies the classification target text 19;
FIG. 7 is a diagram for explaining the contents of processing performed when an assignment suitability question or the like is displayed when one or more assignment candidate categories are determined based on the result of the system dictionary comparison / collation in S13.
FIG. 8 is a view for explaining a case where a user dictionary is updated when an assignment candidate category is not selected for a classification target text 19;
[Explanation of symbols]
1 Text classifier
3 Text group storage for learning
5 Dictionary creation device
7 System dictionary storage
9 User dictionary storage
11A, 11B, ... User dictionary
13 System dictionary
15 Text Classification Unit
17 User dictionary update unit
19 Classification target text

Claims (10)

A system for creating an electronic dictionary for document classification,
A plurality of sample documents belonging to each of a plurality of categories of the document are input, and for each of the plurality of character strings included in the plurality of sample documents, the degree of association between the character strings and the category is determined. Means for determining the degree of association,
Based on the result of the determination, select each specific character string whose degree of association is equal to or higher than a predetermined degree from among the many character strings, and select each of the selected specific character strings and one or more categories corresponding thereto. And a dictionary creating means for creating an electronic dictionary including the following. The association degree determination means may include, for each of the plurality of character strings,
(1) the total number of the character strings included in the plurality of sample documents belonging to the same category;
(2) The dictionary creation system according to claim 1, wherein the degree of association of the character string is determined based on the total number of existing sample documents or the total number of the character strings included in the existing sample document. . A system for classifying documents,
Dictionary acquisition means for acquiring an electronic dictionary in which character strings corresponding to each of a plurality of categories of the document are described;
A document acquisition unit for acquiring a document to be classified;
Each character string included in the acquired document, a comparison and collation unit that performs comparison and collation with each character string described in the electronic dictionary,
As a result of the comparison and collation, when there is a character string that matches each other, the category corresponding to the matched character string is grasped by referring to the electronic dictionary, and the grasp is performed on the acquired document. A document classification unit that classifies the acquired document by assigning an assignment target category that is a category. Ground character string extracting means for extracting the matched character string or the first long character string including the character string that became the ground to be given the target category from the acquired document,
Notifying means for notifying a predetermined user of the extracted matched character string or the first long character string, and the assignment target category;
A question output unit that outputs to the user a question as to whether or not to give the notified category to be given to the document,
Further comprising: answer receiving means for receiving an answer to the output question from the user, wherein the document classifying means only receives a positive answer from the answer receiving means for the acquired document. The document classification system according to claim 3, wherein a category to be provided is provided. The basis character string extracting means includes a second sample including the matched character string from a specific sample document including the matched character string included in a plurality of sample documents belonging to each of the plurality of categories. Extracting a long character string from the specific sample document,
The document according to claim 4, wherein the notifying unit notifies the user of the first long character string, the category to be provided, the second long character string, and a category to which the specific sample document belongs. Classification system. A method for creating an electronic dictionary for document classification,
A plurality of sample documents belonging to each of a plurality of categories of the document are input, and for each of the plurality of character strings included in the plurality of sample documents, the degree of association between the character strings and the category is determined. Performing a determination of
Based on the result of the determination, a plurality of specific character strings whose degree of association is equal to or more than a predetermined degree are selected from among the many character strings, and each of the selected specific character strings and one or more corresponding thereto are selected. Creating an electronic dictionary in which categories are recorded. A method for classifying documents, wherein
Obtaining an electronic dictionary that is a character string corresponding to each of a plurality of categories of the document and includes two or more character strings characteristic of the category;
Obtaining a document to be classified;
Comparing and comparing each character string included in the document with each characteristic character string described in the electronic dictionary;
As a result of the comparison and collation, when there is a character string that matches each other, the category corresponding to the matched character string is grasped by referring to the electronic dictionary, and the grasp is performed on the acquired document. Classifying the acquired document by assigning an assignment target category that is a category. A computer program for creating an electronic dictionary for document classification,
A plurality of sample documents belonging to each of a plurality of categories of the document are input, and for each of the plurality of character strings included in the plurality of sample documents, the degree of association between the character strings and the category is determined. Performing a determination of
Based on the result of the determination, a plurality of specific character strings whose degree of association is equal to or more than a predetermined degree are selected from among the many character strings, and each of the selected specific character strings and one or more corresponding thereto are selected. Creating an electronic dictionary in which categories are recorded, and a computer program for causing the computer to execute the steps. A computer program for classifying documents,
Obtaining an electronic dictionary which is a character string corresponding to each of a plurality of categories of the document, and in which two or more character strings characteristic of the category are described;
Receiving documents to be classified and input;
Performing a comparison and comparison between each character string included in the acquired document and each characteristic character string described in the electronic dictionary;
As a result of the comparison and collation, when there is a character string that matches each other, the category corresponding to the matched character string is grasped by referring to the electronic dictionary, and the grasp is performed on the acquired document. Categorizing the acquired document by assigning an assignment target category, which is a category, to the computer. An electronic dictionary for document classification,
An electronic dictionary corresponding to each of a plurality of categories of a document and including one or more character strings considered or estimated to be characteristic for the category.

Images