JP2005182218A - Dictionary editing device, and document classifying device and its program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a document classifying device for automatically and accurately classifying a document to a proper person in charge. <P>SOLUTION: This document classifying device(mail analyzing device) 2 is provided with a dictionary 4 constituted of a significance dictionary 41 in which a tf/idf value is stored for each word appearing in the category and a simultaneous appearance dictionary 42 in which an idf/conf value is stored for each of the combination of the first word and the second word for the word appearing in the category. The document classifying device 2 is configured to calculate the tf/idf value and idf/conf value of each word of the dictionary by collating the word appearing in an inputted document in the dictionary 4, and to calculate the scores of each category on the basis of the scores of each word calculated by performing a predetermined arithmetic operation based on them, and to classify the inputted document into any of the plurality of categories on the basis of this. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a dictionary editing apparatus, a document classification apparatus, and a program thereof, and more particularly, to a dictionary editing apparatus and document classification for automatically and accurately classifying an electronic document input by e-mail, Web, or the like to an appropriate person in charge. The present invention relates to an apparatus and a program thereof.

  In recent years, there has been an increasing need for responding to inquiries from customers by e-mail or the Web, regardless of telephone or FAX, in a call center of a company. However, in order to respond to an inquiry (inquiry mail) by e-mail or the like with a small number of people, it is often necessary to classify and answer the inquiry mail to an appropriate person in charge. In order to provide a rich service by e-mail or Web, it is important to improve the efficiency of such classification work, and a system for automatically classifying this is strongly desired.

  As a means for automatically classifying such e-mails, there is known an apparatus that learns and generates analogy rules for classifying e-mails from sample data and can automatically classify e-mails based on the learning rules. (For example, refer to Patent Document 1).

On the other hand, in many studies on document classification, tf · idf values (term frequency times inverse document frequency), which are the importance of words alone, are applied. Furthermore, it has been proposed by the present inventors to classify documents such as e-mails using an idf / conf value representing the co-occurrence between two words in addition to the tf · idf value (non-patent document). Reference 1).
JP 2001-256251 A Narita et al. (F-67) Automatic email distribution by text mining and reinforcement learning, 2002 Hokuriku Branch Association

  As described above, the present inventors have previously proposed to classify documents such as e-mails using the tf · idf value and idf / conf value. However, according to the inventor's examination after that, the document classification using only the tf / idf value and idf / conf value does not provide practical classification accuracy when used for e-mail classification. I understood. That is, if one dictionary is created and used using tf · idf value and idf / conf value together as one parameter, a practical classification result cannot be obtained, while tf · idf value and It was found that creating and using two dictionaries using idf / conf values as two separate independent parameters is effective in realizing a practical classification. In addition, it is effective to learn word weights after creating a dictionary, but it is practical to learn two separate independent parameters, tf · idf value and idf / conf value. It was found to be effective in realizing a realistic classification. It was found that it is effective to perform three types of learning (important words, specific words, unnecessary words) in this learning process.

  It is an object of the present invention to provide a dictionary editing apparatus that creates a dictionary for automatically and accurately classifying a document to an appropriate person in charge.

  It is another object of the present invention to provide a document classification device that automatically and accurately classifies a document to an appropriate person in charge.

  It is another object of the present invention to provide a document classification program that automatically and accurately classifies a document to an appropriate person in charge.

  The dictionary editing apparatus of the present invention is a dictionary editing apparatus that creates a dictionary based on a document in which a category belongs, and for each word that appears in the category, a tf · idf value that represents the importance of the word alone is obtained. An importance dictionary to be stored is created, and a co-occurrence dictionary for storing an idf / conf value representing co-occurrence between the words is created for each combination of a plurality of words with respect to the words that appear in the category.

  The document classification apparatus according to the present invention includes a plurality of category dictionaries provided for each of a plurality of categories, and each category dictionary stores a tf · idf value representing the importance of a single word for each word that appears in the category. A dictionary comprising an importance dictionary and a co-occurrence dictionary storing idf / conf values representing co-occurrence between two words for each combination of the first word and the second word for the words appearing in the category; Using the words that appear in the input document, the words are compared in the dictionary to obtain tf · idf values and idf / conf values for each word in the dictionary, and a predetermined calculation is performed based on these values. A document classification device that calculates a score for each word, calculates a score for each category based on the score for each word, and classifies the input document into one of the plurality of categories based on the score Consists of.

  The program of the present invention is a program for realizing a document classification device, and the program includes a plurality of category dictionaries provided for each of a plurality of categories using words appearing in an input document in a computer. An importance dictionary that stores tf · idf values representing the importance of each word for each word that appears in the category, and a combination of the first word and the second word for the word that appears in the category Each word is collated with a dictionary composed of a co-occurrence dictionary that stores idf / conf values representing co-occurrence between two words every time, and the tf · idf value and idf / conf value for each word in the dictionary A process for calculating a score for each word by performing a predetermined calculation based on the tf · idf value and idf / conf value for each word, and the category based on the score for each word A process of calculating the score, a document that was the input based on the scores for each of the categories to perform the processing for classifying the one of the plurality of categories.

  According to the dictionary editing apparatus of the present invention, the tf · idf value representing the importance of a single word and the idf / conf value representing the co-occurrence between a plurality of words are used as two independent independent parameters. Individual dictionaries (tf / idf dictionaries, idf / conf dictionaries) can be created. By using this as a dictionary for document classification devices, practical classification is possible when used for classification of documents such as e-mails. Accuracy can be obtained. Therefore, it is possible to easily create a dictionary for automatically and accurately classifying a document to an appropriate person in charge.

  According to the document classification apparatus of the present invention, by using the above-described two dictionaries tf / idf dictionary and idf / conf dictionary, the input document basically has a tf · idf value representing the importance of a single word. Since it can be classified into categories based on the idf / conf value representing the co-occurrence between multiple (two) words, practical classification accuracy can be obtained when used for classification of documents such as e-mails. . Accordingly, it is possible to automatically and accurately classify the document into an appropriate person in charge.

  According to the document classification program of the present invention, the program is supplied by being stored in a medium such as a flexible disk, a CD-ROM, a CD-R / W, a DVD, or downloaded via a network such as the Internet. Thus, the above-described document classification system can be easily realized, and accurate document classification can be realized.

  FIG. 1 is a block diagram of the document classification system, showing the configuration of the document classification system of the present invention. The document classification system includes three subsystems, that is, a dictionary editing device 1, a document classification device 2, and a reinforcement learning device 3. These are connected to each other by, for example, a LAN (Local Area Network). In this example, the document classification device 2 includes a mail analysis device 2, for example. Accordingly, in the following example, the input document to be classified (analyzed) is, for example, an e-mail 8, the category is a person in charge (respondent) who should distribute the document, and the document classification device 2 Distributes (distributes) the input document to the classified person in charge.

  The dictionary editing device 1 is a dictionary editing device that creates a dictionary based on a document file 6 and stores an importance dictionary that stores tf · idf values representing the importance of a single word for each word that appears in the person in charge. (Tf / idf dictionary) 41 is created, and a co-occurrence dictionary storing idf / conf values representing the co-occurrence between the words for each combination of a plurality of words with respect to the words appearing in the person in charge (Idf / conf dictionary) 42 is created. Since the document file 6 is a document created by each person in charge in daily work (stores the document), the document file 6 is a document to which the person in charge, that is, the person in charge knows. Therefore, the word that appears in the person in charge is a word that appears in the document file 6 created by the person in charge. In this example, the co-occurrence dictionary 42 stores an idf / conf value representing the co-occurrence between two words for each combination of the first word and the second word for the word that appears in the person in charge. Accordingly, the dictionary 4 includes a plurality of person in charge dictionaries 40 provided for each of a plurality of persons in charge, and each person in charge dictionary 40 includes an importance dictionary 41 and a co-occurrence dictionary 42.

  The dictionary editing apparatus 1 actually reads a text file obtained by converting various documents stored in a document file (folder for each person in charge) 6, and removes line breaks and spaces from the read text data. The words extracted by dividing into words by a known morphological analysis process are registered in the dictionary 40 for each person in charge. In the extracted word, the part of speech is assumed to be a particle, auxiliary verb, conjunction, prefix, adverb, adjunct, exchanging verb, and symbol, and are not registered in the dictionary 4 because they are considered unnecessary words.

  That is, in the present invention, in the creation and editing of the dictionary 4, the tf · idf value that is the importance of each word is used, and the co-occurrence between words is also used as an idf / conf value (idf divided by confidence). . Here, the tf value indicates the frequency of words appearing in the document (ie, the importance of the word), the idf value indicates how much the word appears in a specific classification (ie, the word specificity), and tf・ The idf value is used in the study of document classification and indicates the weight that takes into account the importance and specificity of the word (that is, the importance of the word alone), and the conf value indicates the co-occurrence of the word The confidence level is shown, and the idf / conf value shows the co-occurrence between highly specific words.

  FIG. 2A shows an example of a dictionary (importance dictionary) 41 of tf · idf values. The importance dictionary 41 stores the tf value, idf value, tf · idf value for each word a1, a2, a3,... Used by the person in charge. Calculation of the tf value, idf value, and tf · idf value will be described later. FIG. 2B shows an example of a dictionary (co-occurrence dictionary) 42 of idf / conf values. For each combination of the first word and the second word in the words b1, b2, b3,... Used by the person in charge, the simultaneous appearance dictionary 42 has the idf value of the first word, the first word and the second word. Conf value between, and idf / conf value between the first word and the second word are stored. As described above, the idf / conf value in this example indicates co-occurrence considering the specificity between the two words. The calculation of the conf value and idf / conf value will be described later.

  The mail analysis device 2 collates the words in the dictionary 4 using words appearing in the inquiry e-mail (inquiry mail) 8 that is the input document, and determines the tf · idf value and idf for each word in the dictionary 4. / conf value is calculated, based on these, the score for each word is calculated, the score for each person in charge is calculated based on the score for each word, and the document input based on this is calculated. Classify it as one of multiple personnel. In this example, the mail analysis device 2 calculates the product of the tf · idf value for each word in the dictionary 4 and the match rate between the idf / conf value and the word appearing in the input document as the predetermined calculation. To calculate a score for each word. Further, the mail analysis device 2 classifies the inquiry mail (hereinafter simply referred to as mail) 8 that is an input document into a predetermined number of persons in charge from the top, based on the score of each person in charge.

  Similarly to the dictionary editing device 1, the mail analysis device 2 performs the well-known morphological analysis processing on the body of the input mail 8 after removing line breaks and spaces, and obtains the word obtained by dividing the word into words. The dictionary 4 is checked. At this time, the mail analysis device 2 calculates the scores of all persons in charge for the mail 8 from the tf / idf value, the idf / conf value, and the word match rate for the matched word or word combination. Finally, one or a plurality of persons in charge who have a high score are estimated as respondents to the mail 8 (persons who should answer the mail 8), and the result is, for example, mail as shown in FIG. Displayed on the analysis device 2. That is, the mail 8 from the interrogator is input, and the candidate for the person in charge (that is, FIG. 3) to be answered is obtained as the output. Further, in this example, the mail analyzing apparatus 2 sends the inputted mail 8 to the person in charge (respondent) terminal 5 of the person in charge in FIG. 3 according to the classification result (score). Is distributed (distributed).

  FIG. 3 shows an example of the mail analysis result. The result consists of a score score (d), a word t that can be collated between the mail 8 and the dictionary, and a score (d, t) for each word for each person in charge d. . The person in charge d and the matching word t are displayed in descending order of score. The one with the highest score (first (i) position) is called the first (i) position collation word. In FIG. 3, the person in charge with “◯” is the respondent. In this example, the top two people are estimated as respondents. The input (received) mail 8 is classified (transmitted) as the mail 8 to the person in charge who is the respondent. On the other hand, the person in charge with “x” is not estimated as a respondent. In the case of interposing a human classifier or when the person in charge who has received the classification reclassifies to another person in charge, the classification may be performed with reference to the estimation result shown in FIG.

  Although not shown, the mail analysis device 2 is connected to the respondent terminal 5 via a LAN, for example, and is connected to a mail server. The mail server is connected to the Internet, receives mail 8 from the outside, and sends it to the mail analysis device 2 for distribution to each terminal 5 inside. The e-mail 8 is created (input) by inputting the inquiry content, name, e-mail address for reply, etc. using, for example, a Web browser known to the questioner.

  In the present invention, the two weights obtained based on the two dictionaries 41 and 42 are not used as they are, but the score is calculated in consideration of the matching rate of matched (matched) words. That is, the words appearing in the mail 8 to be processed and the words in the dictionary are not matched completely but partially matched, and the partial match rate (word match rate) is used for score calculation. Thereby, the influence of a slight individual difference etc. about the same term can be excluded.

  The reinforcement learning device 3 determines the tf · idf value in the importance dictionary 41 once created based on the result of classifying a predetermined document (that is, another document file 7) into one of a plurality of persons in charge by the mail analysis device 2. And the idf / conf value in the co-occurrence dictionary 42 is updated (enhanced). That is, the reinforcement learning device 3 performs important word learning, specific word learning, and unnecessary word learning. In the important word learning process, when a predetermined document is classified as a true person in charge, a word that appears in the predetermined document and is collated in the dictionary 40 of the true person in charge is regarded as an important word. Increase the weight. In the specific word learning process, when a predetermined document is not classified as a true person in charge, among words appearing in the predetermined document, a word having high specificity for the true person is re-evaluated. , Increase the weight of the specific word of the true person in charge. Unnecessary word learning is performed when a given document is classified as a person other than the true person, and the person who appears in the given document and is classified incorrectly (in charge other than the true person) A word in the dictionary 40 of the person in charge (category) is regarded as an unnecessary word, and the weight in the dictionary 40 of the person in charge classified incorrectly is reduced, and the weight in the dictionary 40 of persons in charge other than the true person in charge is reduced. . Thereby, the classification accuracy of the mail 8 can be further improved.

  In the present invention, in order to reinforce the tf · idf value and idf / conf value in the dictionary 4 once created, a separate document file 7, that is, a document created by each person in charge who receives the classification in various tasks is created. Use the read file. The separate document file 7 is a separate document (document file) from the document file 6 and the mail 8 and is a separate document created by each person in charge (stores the separate document). In other words, the separate document file 7 is a file in which a document created by a person in charge at any time is taken in and stored. The separate document file 7 is classified into one of a plurality of persons in charge by the mail analysis device 2 in the same manner as the document file 6. Since the person in charge of the separate document file 7 is also clear, this result can be evaluated accurately, and the tf / idf value and idf / conf value can be strengthened as needed according to the changes in the work contents of the person in charge. This is effective for accurate classification of the mail 8.

  In the present invention, important word learning, specific word learning, unnecessary word learning, that is, enhancement of tf · idf value and idf / conf value is performed by profit sharing (hereinafter referred to as PS). That is, the person in charge (classification destination) of the separate document file 7 is estimated, and the tf · idf value and idf / conf value are corrected by the reward (profit) according to the appropriateness of the estimation result. PS is attracting attention in reinforcement learning. This makes it possible to obtain a classification accuracy equivalent to that of a classification specialist. As is well known, PS reinforces a plurality of rules (a weight of a plurality of words in the present invention) at a time when a reward is obtained, so that the classification accuracy of the mail 8 can be improved efficiently. In addition, a system that incorporates reinforcement learning into document classification in this way has not been developed so far.

  As described above, according to the present invention, a practical classification can be realized by classifying a document such as the mail 8 using two weights obtained based on the two dictionaries 41 and 42. it can. That is, the classification result (classification accuracy) of an expert who classifies the mail 8 as a daily work can be considered as a practically necessary accuracy. As the classification accuracy by the mail analysis device 2 of the present invention, it is possible to obtain an accuracy substantially equal to the classification accuracy of the expert. Therefore, the mail analysis apparatus 2 (classification by the present invention) of the present invention can sufficiently withstand practical use. According to the study of the present inventor, the classification accuracy using only the two dictionaries 41 and 42 is actually slightly lower than the accuracy of the expert, but the tf / idf value and idf / conf value are stored in a separate document file. By performing reinforcement learning using 7, the mail 8 can be classified with the same accuracy as a classification specialist.

  Hereinafter, the processing in the document classification system of the present invention will be described in detail with reference to the processing flow. FIG. 4 is a document classification processing flow, and shows the document classification processing in the document classification system shown in FIG. 1 of the present invention. The dictionary editing device 1 collects the document files 6 created by each person in charge, and the weights of the words appearing therein are expressed as tf · idf values representing the importance of the words alone and the co-occurrence between the two words. These are calculated as idf / conf values, and these two types of dictionaries 41 and 42 are created for each person in charge (step S101). The reinforcement learning device 3 applies PS to reinforcement learning of these weights or weights (indicating tf · idf value and idf / conf value) (step S102). That is, the mail analysis device 2 determines whether or not the mail 8 has been received, and if there is no such reception, the document in the separate document file 7 is checked against the two types of dictionaries 41 and 42 to match the word weight with the word. From the rate, a score is calculated for each person in charge, a person in charge with a high score is estimated as a respondent to the mail 8, and the result of the estimation is input to the reinforcement learning device 3, and the tf · idf value and idf Update the weight of the / conf value. On the other hand, when the mail 8 is received, the mail analysis device 2 analyzes the mail 8 (step S103). That is, the mail 8 and the two types of dictionaries 41 and 42 are collated, and a score is calculated for each person in charge from the word weight and the word match rate. Then, the mail analysis apparatus 2 estimates a person in charge with a high score as a respondent to the mail 8, and displays this on a terminal (not shown) of the mail analysis apparatus 2 (step S104).

  FIG. 5 is a dictionary editing process flow, and shows the dictionary editing process executed by the dictionary editing apparatus 1 of FIG. 1 in step S101. The dictionary editing device 1 reads all document files 6 of all persons in charge (step S111), and preprocesses all documents (step S112). That is, the line break and space are removed from the entire document, and the remaining part is subjected to a well-known morphological analysis to obtain a separated word. Then, unnecessary part-of-speech words are deleted from all the obtained words. Next, the dictionary editing apparatus 1 generates a person-in-charge / word table shown in FIG. 2C (step S113), calculates tf / idf values based on the table (step S114), and calculates the word and tf The idf value is written into the tf / idf dictionary (importance dictionary) 41 (step S115). As a result, the tf / idf dictionary 41 is created. The person-in-charge / word table and its generation will be described later with reference to FIG. Next, the dictionary editing apparatus 1 generates a word combination table shown in FIG. 8 (step S116), calculates an idf / conf value based on the table (step S117), and calculates the word, its combination, and idf / The conf value is written into the idf / conf dictionary (co-occurrence dictionary) 42 (step S118). Thereby, the idf / conf dictionary 42 is created. The word combination table and its generation will be described later with reference to FIG.

  FIG. 6 is a flowchart of the person-in-charge / word table generation process, and shows the person-in-charge / word table generation process executed by the dictionary editing apparatus 1 in step S113. The dictionary editing apparatus 1 sets the first person in charge to d (step S121), adds d to the relevant column of the person in charge / word table in FIG. 2C, and sets all the elements tf (t, d) to “ Is initialized to 0 (step S122), the first word is set to t (step S123), and it is checked whether or not t exists in rows t1, t2, t3,. S124). If it does not exist, the dictionary editing apparatus 1 adds t to the line of the person-in-charge / word table, and initializes all the elements tf (t, d) to “0” (step S125). If it exists, the dictionary editing apparatus 1 omits step S125 and adds “1” to the elements corresponding to the person in charge d and the word t (step S126). That is, tf (t, d) = tf (t, d) +1 is executed. Thereafter, the dictionary editing apparatus 1 checks whether or not the processing has been completed for all the words of the person in charge d (step S127), and if not, sets the next word to t (step S128), and step Repeat S124 and subsequent steps. If it has been completed, the dictionary editing apparatus 1 checks whether or not processing has been completed for all persons in charge (step S129). If it has not been completed, the next person in charge is set in d (step S1210). Step S122 and subsequent steps are repeated. If the process has been completed, the process ends. As a result, a person-in-charge / word table is generated.

  Now, the tf · idf value is defined as in equation (1). That is,

Here, tf (t, d) in the first term on the right side represents the number of appearances of the word t in all documents (document file 6) created by the person in charge d. Therefore, the first term on the right side gives a large weight to a word used repeatedly by the same person in charge. For example, in FIG. 2C, the number of appearances tf (t1, d1) = 1 of the word t1 in the person in charge d1. The denominator of the first term on the right side represents the sum of the frequencies of words used by the person in charge d. For example, in FIG. 2C, the value Σtf (t ′, d1) = 8538 for the person in charge d1. N in the second term on the right side represents the number of all persons in charge as classification destinations, and df (t) represents the number of persons in charge using the word t. Therefore, the second term on the right side gives a large weight to the words used by a small number of persons in charge, and serves as an index of specificity that characterizes each person in charge. From the above, the tf value (tf _t ^d ), idf value (idf _t ^d ), and tf · idf value can be calculated based on the person-in-charge / word table.

FIG. 7 is a word combination table generation processing flow, and shows the word combination table generation processing executed by the dictionary editing apparatus 1 in step S116. Dictionary editing apparatus 1 reads the minimum support and minimum confidence (step S131), set the first person to d (step S132), generates a sentence set S ^d by the search period (step S133 ), and it calculates the LSC and a minimum total sentence number of support of the sentence set S ^d (step S134). Next, the dictionary editing apparatus 1 sets the number of word combinations to “1”, that is, sets the word combination table (hereinafter the same) C = C1 (step S135), (combination of C words) = (sentence set S duplicate without words) and in each sentence extracted from ^d (step S136), the sentence number of the sentence set S ^d the count value of the C (count) a combination of words C appears (step S137). Next, the dictionary editing apparatus 1 checks whether or not the count value of C for the combination is greater than or equal to LSC for all the combinations of words in C, and if not, removes the combination of the words from C If so, the combination of the words is not removed from C (step S138). Thereafter, the dictionary editing apparatus 1 checks whether the number of word combinations is “2” (step S139). When it is not “2”, that is, when the number of word combinations is “1”, the dictionary editing apparatus 1 sets the number of word combinations to “2”, that is, C = C2 (step S1310). Combination) = (combination of two words generated from C1) (step S1311), and step S137 and subsequent steps are repeated.

  When the number of word combinations is “2” in step S139, the dictionary editing apparatus 1 sets t1 as the first word of the C2 word combination (step S1312), and sets the count value (t1_Count) of t1 to t1 of C1. The count value (Count) is set (step S1313), and the LCC is calculated from the minimum certainty factor and the count value of C1 of t1 (step S1314). Next, the dictionary editing apparatus 1 checks whether or not the count value of C2 for the combination is greater than or equal to LCC for all the combinations of words in C2, and if not, removes the combination of the words from C2, and so on If not, the word combination is not removed from C2 (step S1315). Thereafter, the dictionary editing apparatus 1 checks whether or not all persons in charge have been processed (step S1316). If not, the next person in charge is set in d (step S1317), and step S133 and subsequent steps are repeated. If all the persons in charge have been processed, the process ends.

Here, the idf / conf value is based on a well-known certainty factor used for extracting an association rule. In order to apply the method of the association rule to the present invention, let T ^d = (t1, t2, t3,..., Tm ⁾ be a set having the words used by the person in charge d as elements. Also, let S ^d = (s1, s2, s3,... Sn) (si⊆T ^d ) be a set of sentences (one sentence delimited by the punctuation marks) extracted from the document created by the person in charge d. Here, the support level (t) of the word t represents the ratio of sentences including t to the entire S ^d . The association rule is expressed as t⇒t ′, which indicates that the probability that the word t ′ appears in the sentence in which the word t appears is high, that is, the co-occurrence of t and t ′ is high. The association rule has two parameters, support and confidence, and these values indicate the significance of the association rule. Here, support (t⇒t ') is t and t with respect to the entire S ^d' and percentage of sentences percentage sentences containing both, confidence (t⇒t ') is the t in the sentence that contains the t' containing Is defined.

  In the present invention, as shown in FIG. 8, a minimum support level and a minimum confidence level are set, a combination of words having high co-occurrence is obtained, and the confidence level is used as a weight representing the co-occurrence level. In this example, the minimum support level and the minimum confidence level were set to 0.3 and 0.7, respectively. These values can be determined empirically. As is well known in research on index word extraction, infrequent words are unnecessary words, but words with higher frequencies are often general words, not feature words, and are unnecessary words. Similarly, a combination of words that does not satisfy the minimum support level and the minimum certainty level is unnecessary and is not registered in the dictionary 4. At the same time, combinations with higher support or certainty are common word combinations and are not necessary. Therefore, as a specific weight representing the co-occurrence of a combination of words satisfying the minimum support level and the minimum confidence level as described above, the idf value of the first word in consideration of the specificity of the word as the reciprocal of the confidence level The value obtained by integrating is used. Furthermore, a value obtained by integrating the idf value of the second word in addition to the first word may be used as the weight, or the support level may be used instead of the certainty factor.

For example, as shown in FIG. 8, it is assumed that the sentence set S ^{d of the} person in charge ^d is obtained. If the number of word combinations is one, each occurrence count Count is obtained (first C1). Now, (minimum support) × except small words {word 4} of Count value than (S ^d number sentences) = 0.3 × 4 (2-th C1). Thus, a value satisfying the minimum support level for each word is determined. Next, for all combinations of two words appearing for the remaining words, the number of occurrences of each count is obtained (first C2), and from this, a combination of words having a count value smaller than 0.3 × 4 {word 1, word 2} and {word 1, word 5} are removed (second C2). Next, for the remaining word combinations, combinations {words 2, words 3} and {words having a Count value smaller than (minimum confidence) × (value satisfying the minimum support for each first word) Except word 3, word 5} (third C2). That is, {word 2, word 3} is 2 (Count value in C2, the same applies hereinafter) <0.7 × 3 (Count value of second C1 word 2 is applied), {word 3, word For 5}, 2 <0.7 × 3, which is excluded. On the other hand, 2> 0.7 × 2 for {word 1, word 3}, and 3> 0.7 × 3 for {word 2, word 5}, which remain.

  According to the study of the present inventor, it has been found that the classification accuracy of the mail 8 (document) is not improved even if the number of word combinations is 3 or more. Accordingly, in this example, word combinations are limited to “2” in order to reduce the amount of calculation. Therefore, in this example, the co-occurrence dictionary 42 stores an idf / conf value for each combination of the first word and the second word.

  Now, an idf / conf value, which is an index with which the word t ′ co-occurs when the word t appears, is defined by equation (2).

Here, the numerator of the second term on the right side is the maximum value (max) of the confidence value (that is, the certainty factor) for a certain person in charge d, and the confidence value having a different size for each person in charge is standardized. From the above, the conf value and idf / conf value can be calculated based on the word combination table of FIG. The definition of the conf value will be clear by removing idf _t ^d from both sides of equation (2).

  FIG. 9 is a flow of mail (inquiry mail) analysis processing, and shows mail analysis processing executed by the mail analysis apparatus 2 of FIG. 1 in steps S103 and S104. The mail analysis device 2 reads the received mail (inquiry mail) 8 to be processed (step S141), removes the line feed and space, and performs a known morphological analysis on the remaining part (step S142). ), Get the word you shared. Next, the mail analysis device 2 sets the first person in charge to d (step S143), sets the first word of the mail 8 to t1 (step S144), and performs tf · idf value weighted addition (step S144). S145), idf / conf value weighted addition is performed (step S146), and it is checked whether or not the processing has been completed for all words in the mail 8 (step S147). The tf · idf value weighted addition will be described later with reference to FIG. 10, and the idf / conf value weighted addition will be described later with reference to FIG.

  If the processing has not been completed for all words, the mail analysis device 2 sets the next word of the mail 8 to t1 (step S148), and repeats step S145 and subsequent steps. If the processing has been completed for all words, the mail analysis device 2 calculates the score of the person in charge d as the sum of the scores for all the words in the mail 8, that is, Score (d) = ΣScore (d, t1) Then, it is checked whether or not the processing has been completed for all persons in charge (step S1410). If the processing has not been completed for all the persons in charge, the mail analyzing apparatus 2 sets the next person in charge for d (step S1411), and repeats step S144 and subsequent steps. When the processing is completed for all persons in charge, the mail analysis device 2 calculates the average and standard deviation of Score (d) for all persons in charge, and based on this, the person in charge who should reply to the mail 8 Candidates (respondents) are determined (step S1412) and displayed (step S1413).

  FIG. 10 is a tf · idf value weighted addition process flow, and shows the tf · idf value weighted addition process executed by the mail analysis device 2 in step S145. The mail analysis device 2 sets (substitutes) Max_tf · idf_Ratio to “0”, sets “0” to Score (d, t1), reads m (step S151), and sets t_dic to tf of the person in charge d The first word of the idf dictionary 41 is set (step S152), and it is checked whether or not t1 and t_dic partially match (step S153). If there is a partial match, the mail analysis device 2 sets the tf · idf value of t_dic to tf · idf, sets the mth power of the match rate between t1 and t_dic to Matched_Ratio, and sets tf · idf and Matched_Ratio to tf · idf. Idf_Ratio is calculated (step S154), and it is checked whether tf · idf_Ratio is larger than Max_tf · idf_Ratio (step S155). If it is larger, the mail analysis device 2 sets tf · idf_Ratio to Max_tf · idf_Ratio (step S156), and checks whether or not processing has been completed for all words in the tf · idf dictionary 41 of the person in charge d (step S156). Step S157).

  If the processing has not been completed for all words, the next word in the tf / idf dictionary 41 of the person in charge d is set in t_dic (step S158), and step S153 and subsequent steps are repeated. If t1 and t_dic do not partially match in step S153, step S154 to step S156 are omitted, and step S157 is executed. If tf · idf_Ratio is not greater than Max_tf · idf_Ratio in step S155, step S156 is omitted and step S157 is executed. When the processing has been completed for all the words in step S157, Max_tf · idf_Ratio is set in Score (d, t1) and the processing is terminated (step S159).

  FIG. 11 is an idf / conf value weighted addition processing flow, and shows the idf / conf value weighted addition processing executed by the mail analysis device 2 in step S145. The mail analysis device 2 sets “0” to Max_idf / conf_Ratio, reads m (step S161), sets the first first word of the idf / conf dictionary 42 of the person in charge d to t_dic1, and sets t_dic2. The first second word in the idf / conf dictionary 42 of the person in charge d is set (step S162), and it is checked whether or not t1 and t_dic1 partially match (step S163). If there is a partial match, the mail analysis device 2 sets the next word after t1 of the mail 8 to t2 (step S164), and checks whether t2 and t_dic2 partially match (step S165). If there is a partial match, the mail analysis device 2 sets the idf / conf values of t_dic1 and t_dic2 in idf / conf, sets the m-th power of the match ratio of t1 and t_dic1 in Matched_Ratio1, and sets t2 and t_dic2 in Matched_Ratio2 The match power of m is set, and idf / conf_Ratio is calculated based on idf / conf, Matched_Ratio1, and Matched_Ratio2 (step S166).

  If t2 and t_dic2 do not partially match in step S165, the mail analysis apparatus 2 checks whether or not the last word of the mail 8 has been processed (step S1611), and if the process has not ended up to the last word, The next word after t2 of the mail 8 is set in t2 (step S1612), and step S165 and subsequent steps are repeated.

  After step S166, the mail analysis device 2 checks whether idf / conf_Ratio is larger than Max_idf / conf_Ratio (step S167). If so, sets idf / conf_Ratio to Max_idf / conf_Ratio (step S167). S168), it is checked whether or not all the words in the idf / conf dictionary 42 of the person in charge d have been processed (step S169). If processing has not been completed for all words, the mail analysis device 2 sets the first word next to the idf / conf dictionary 42 of the person in charge d to t_dic1, and sets the idf / conf dictionary 42 of the person in charge d to t_dic2. The next second word is set (step S1610), and the steps after step S163 are repeated. If the processing has been completed for all words, the mail analysis device 2 reads the constant α (step S1613), and adds the value obtained by α and Max_idf / conf_Ratio to Score (d, t1). d, t1) are calculated (step S1614).

  If t1 and t_dic1 do not partially match in step S163, the mail analysis device 2 omits steps S164 to S168 (including S1611 and S1612) and executes step S169. When idf / conf_Ratio is not larger than Max_idf / conf_Ratio in step S167, the mail analysis device 2 omits step S168 and executes step S169. If the process has been completed up to the last word of the mail 8 in step S1611, the mail analysis apparatus 2 omits step S1612 (including S166 to S168) and executes step S169.

  Thus, in the present invention, all the appearance words in the mail 8 are checked against the dictionary 4 to calculate the score of each person in charge for the mail 8. At this time, the word length of the words divided by the morphological analysis may change. Also, there is no guarantee that the questioner and the person in charge will use the same word. For this reason, if the words included in the mail 8 and the words in the dictionary 4 are collated on the condition of complete matching, the number of matching words decreases, and eventually the classification accuracy does not improve. Therefore, in the present invention, word matching is performed by partial matching, and weighted addition is performed by multiplying the matching rate by the weight of the word to calculate the score of the person in charge d for the mail 8.

  First, a score that takes into account the tf · idf value and idf / conf value of a word t in the mail 8 is defined by equation (3). Note that the word t and the word t ′ indicating co-occurrence with the word t may collate with a plurality of words or combinations of words in the dictionary 4 for the above partial matching.・ Add the idf value and idf / conf value that is the maximum value obtained by multiplying the match rate.

Here, Match_Ratio (t) represents the word matching rate, that is, the ratio of the number of matched characters to the word length of the word t in the mail 8 and the word in the dictionary 4, and is defined by the equation (4).

Here, t _dic represents a word in the dictionary 4, Length (t) represents the number of characters in the word t, and Match_Length (t, t _dic ) represents the number of characters in which t and t _dic match. Further, α in Expression (3) is a coefficient that determines the weight of the first term and the second term. Further, m in the formula (4) represents in which order the word matching rate is reflected in the score. Since this coincidence rate is 1 or less, the larger the value of m, the more the score varies with this rate. It should be noted that both α and m can be varied within an appropriate range, and α and m for obtaining the maximum classification accuracy can be employed based on empirical rules.

Finally, the score Score _t ^d of term t given by equation (3), by taking the sum with words that appear in the mail 8 body, a score representative d to the mail 8 entered by the formula (5) Define.

Here, the respondent to the input mail 8 assumes that the distribution of Score ^d of all the persons in charge is a normal distribution, and assumes that the person in charge has a score that is 2σ (σ is a standard deviation) or more than the average score. . This is because, in the actual mail 8, when the inquiry content is not particularly limited, there are many cases where multiple questions are asked in combination with multiple matters or questions across multiple specialized fields. This is because it is necessary to answer in cooperation.

  FIG. 12 is a flowchart of reinforcement learning processing, and mainly shows the word weight reinforcement learning processing executed in step S102 by the reinforcement learning device 3 of FIG. The reinforcement learning device 3 reads the separate document file 7 (step S171), and extracts a true answerer set (step S172). The separate document file 7 includes, for example, a learning mail 8 (the same applies hereinafter). The mail analysis device 2 performs mail analysis on the separate document file 7 as described above, and obtains the result of FIG. The reinforcement learning device 3 acquires the result as an email analysis result table (step S173), and checks whether the respondent is included in the true answerer set for all the respondents in the email analysis result table. (Step S174) When it is included, the important word learning process is executed for the respondent (step S175), and when it is not included, the unnecessary word learning process is executed for the respondent (step S176). In practice, as indicated by the dotted line, the execution of steps S174 and S175 or S176 for one respondent is repeated for each respondent. Thereafter, the reinforcement learning device 3 checks whether or not the non-respondent is included in the true respondent set for all non-responders in the mail analysis result table (step S177). The specific word learning process is executed for the non-responder (step S178), and if not included, step S178 is omitted. In practice, as indicated by the dotted line, the execution of steps S177 or S177 and S178 for one non-responder is repeated for each non-responder. Thereafter, the reinforcement learning device 3 checks whether or not all the separate document files 7 have been processed (step S179), and if not processed, repeats step S171 and subsequent steps, and ends the processing if processed. To do.

  In the present invention, the analysis result of the separate document file 7 is received and the tf · idf value and idf / conf value of the dictionary 4 for each person in charge are updated by the PS based on the estimated correctness of the respondent. In this example, the separate document file 7 whose creator, that is, the person in charge is known is input to the mail analysis device 2. In this case, since the true document creator is known, the correctness of the classification result of the system can be determined. Note that the result of classifying the separate document file 7 with reference to a human classifier may be learned as correct.

  In the present invention, the following three types of reinforcement learning are performed. That is, in the learning of the important word, when the true respondent can be estimated that the system is the respondent, the word collated in the separate document file 7 and the respondent's dictionary 40 is regarded as the important word, and the weight is increased. To do. In addition, when learning a specific word, if a true respondent cannot be estimated as a respondent, the word that appears in the separate document file 7 is re-examined as a highly specific word for the true respondent. By evaluating, the weight of the specific word of the respondent is increased. Furthermore, unnecessary words are learned in the case where a person other than the true respondent mistakenly estimates that the system is the respondent, and the person who is collated in the separate document file 7 and the dictionary 40 of the person in charge is regarded as an unnecessary word. The weight of unnecessary words in the dictionary 40 is reduced. At the same time, this word is also learned as an unnecessary word in the dictionary 40 of the person in charge other than the true respondent.

  FIG. 13 is a flowchart of important word learning processing, showing the important word learning processing executed by the reinforcement learning device 3 in step S175. The reinforcement learning device 3 reads the constants c1, L, and S from the definition file (step S181), and sets the first matching word of the target respondent in the mail analysis result table to the word t (step S182). Next, the reinforcement learning device 3 searches for t in the target respondent's tf • idf dictionary 41 and sets tf • idf for the word matched with t in tf • idf, and uses f_tf from tf • idf and c1.・ Idf is calculated (step S183), t is searched with the first word in the target respondent's idf / conf dictionary 42, idf / conf is set to idf / conf and the idf / conf value of the word matched is set to idf / conf F_idf / conf is calculated from c1 and c1 (step S184), and the tf · idf value and idf / conf value of t in the tf · idf dictionary 41 and idf / conf dictionary 42 are updated (step S185). That is, the new tf · idf value is a value obtained by adding f_tf · idf to the previous tf · idf value. The new idf / conf value is the value obtained by adding f_idf / conf to the previous idf / conf value.

  Next, the reinforcement learning device 3 sets i = 2 (step S186), and sets the i-th collation word of the target respondent in the mail analysis result table to the word t (step S187). Next, the reinforcement learning device 3 searches for t in the target respondent's tf • idf dictionary 41 and sets tf • idf value of the word matched with t in tf • idf, and newly adds f_tf • idf and S F_tf · idf is calculated (step S188), t is searched with the first word in the target respondent's idf / conf dictionary 42, idf / conf value of the word matched with t is set in idf / conf, and f_idf New f_idf / conf is calculated from / conf and S (step S189), and the tf · idf value and idf / conf value of t in the tf · idf dictionary 41 and idf / conf dictionary 42 are updated (step S1810). That is, the new tf · idf value is the value obtained by adding f_tf · idf to the previous tf · idf value, the new idf / conf value is the value obtained by adding f_idf / conf to the previous idf / conf value And Thereafter, the reinforcement learning device 3 checks whether i is equal to L (step S1811). If it is not equal, i is set to i + 1 (step S1812), and step S187 and subsequent steps are repeated. If equal, step S1812 is omitted and the process ends. Note that if the text of the mail 8 is extremely short, the maximum value of i may be smaller than L. In this case, the maximum value of i is used instead of the read L. That is, if i is equal to the maximum value of i in step S1811, the process is terminated (hereinafter the same in steps S1911 and S2120).

In PS, the weight added to the rule is strengthened for each episode. An episode represents a rule sequence from the initial state or immediately after obtaining a reward to the next reward. For this strengthening, a strengthening function with an argument of how far in the past from the reward is used. Episodes of length l with respect to (rl, ·· ri, ·· r2 , r1), the weights w _i of the rule r _i is reinforced with reinforcing function f _i as in Equation (6).

In the present invention, a rule corresponds to a word or a combination of words in the dictionary 4, and a rule series of episodes is a word (tl,... Ti,... T 2, t 1) whose score Score ^d is determined by the equation (5). is there. That is, the words collated in the separate document file 7 and the dictionary 40 of the person in charge d. Furthermore, the order i in the sequence the contribution to the score Score ^d of formula (5) forward, i.e. formula (3) and the size of the order of Score _t ^d, the score position 1 of the Score _t ^d is maximum The word t (first collation word) is assumed to be t1. Further, in the present invention, the word weight is composed of two types of tf · idf value and idf / conf value, so that the equation (6) becomes the equation (7) and the equation (8), respectively. When the respondent estimated by the system matches the true respondent, the higher the i is, the higher positive reward is given to the word ti of the person in charge d.

However, in the learning method of the present invention, the tf · idf value and the idf / conf value are enhanced separately, but the method is exactly the same (see the processing in FIG. 13).

  In PS, it must be ensured that valid word weights are enhanced and invalid word weights are suppressed. It is well known that this condition satisfies the expression (9) by the rationality theorem.

Here, W is the maximum length of an episode, L is the maximum number of valid rules that exist under the same sense input, and in the present invention, the number of words that enables learning is limited, and is matched between the separate document file 7 and the dictionary 4 The number of words in the score Score ^d to be learned is set to L, and the number of words is set to L (for example, if L = 10, the top 10 words are learned, and so on). The simplest enhancement function that satisfies the theorem of Equation (9) is the geometric ratio decreasing function represented by Equation (10).

Here, as is well known, S ≧ L + 1 must be satisfied. Further, the initial value f ₁ ^d of equation (10), that is, the reinforcement value of the word having the maximum score Score _t ^d is defined by the following equation (11). Here, c1 is a constant.

As described above, learning of the important word is performed when the estimated respondent is a true answerer, and a positive reward is given to the weight of a word having a higher score Score _t ^d and the weight is increased.

  FIG. 14 is an unnecessary word learning process flow, and shows the unnecessary word learning process executed by the reinforcement learning device 3 in step S176. The reinforcement learning device 3 reads the constants c2, L, and S from the definition file (step S191), and sets the first matching word of the target answerer in the mail analysis result table to the word t (step S192). Next, the reinforcement learning device 3 searches the target respondent's tf • idf dictionary 41 for t, sets tf • idf to tf • idf and sets the tf • idf value of the word matched with t, and uses f_tf from tf • idf and c2・ Idf is calculated (step S193), t is searched with the first word in the target respondent's idf / conf dictionary 42, idf / conf is set to idf / conf, and idf / conf is set to idf / conf. F_idf / conf is calculated from c2 and c2 (step S194), and unnecessary word reinforcement processing is executed (step S195). The unnecessary word enhancement processing will be described later with reference to FIG.

  Next, the reinforcement learning device 3 sets i = 2 (step S196), and sets the i-th collation word of the target answerer in the mail analysis result table to the word t (step S197). Next, the reinforcement learning device 3 searches for t in the target respondent's tf • idf dictionary 41 and sets tf • idf value of the word matched with t in tf • idf, and newly adds f_tf • idf and S F_tf · idf is calculated (step S198), t is searched with the first word in the target respondent's idf / conf dictionary 42, idf / conf value of the word matched with t is set in idf / conf, and f_idf New f_idf / conf is calculated from / conf and S (step S199), and unnecessary word reinforcement processing similar to step S195 is executed (step S1910). Thereafter, the reinforcement learning device 3 checks whether i is equal to L (step S1911). If not, i is set to i + 1 (step S1912), and step S197 and subsequent steps are repeated. If equal, step S1912 is omitted and the process ends.

  FIG. 15 is a flow of unnecessary word reinforcement processing, and shows the reinforcement processing of unnecessary words executed by the reinforcement learning device 3 in steps S195 and S1910. The reinforcement learning device 3 sets a target respondent to the person in charge d (step S201), and checks whether tf · idf−f_tf · idf is greater than “0” (step S202). The tf · idf value of t in the dictionary 41 is updated to tf · idf −f_tf · idf (step S203). If not large, the tf · idf value of t in the tf · idf dictionary 41 is updated to “0” (step S203). S204). Next, the reinforcement learning device 3 checks whether or not idf / conf−f_idf / conf is larger than “0” (step S205). If it is larger, the idf / conf value of t in the idf / conf dictionary 42 is set to idf / Update to conf-f_idf / conf (step S206), and if not, update the idf / conf value of t in the idf / conf dictionary 42 to "0" (step S207). Next, the reinforcement learning device 3 checks whether or not all persons in charge other than the true answerer set have been processed (step S208). If not processed, the person in charge d is assigned to the next person in charge (target answerer). (Step S209), t is searched in the tf · idf dictionary 41 of the person in charge d, and tf · idf value of the word matched with t is set in tf · idf (step S2010). The first word in the / conf dictionary 42 is searched for t, idf / conf value of the word matched with t is set in idf / conf (step S2011), and step S202 and subsequent steps are repeated. If all the persons in charge have been processed in step S208, the process ends.

  Unnecessary words are learned when it is incorrectly estimated that the system is the answerer except for the true answerer. In this case, the estimated upper word of the respondent is originally an unnecessary word, but it is considered that a large weight is given. Therefore, a negative reward is given to the weight of such a word, and the weight is reduced by Expression (12).

Note that the weight is prevented from being negative in the equation (12). Further, the geometric ratio function of the reinforcement value is the same as that in the equation (10), and the initial value is given by the equation (13).

Here, c2 is a constant. Furthermore, the majority of such unnecessary words are empirically found in general terms, and not only those who the system estimates as respondents, but also by giving negative rewards in the dictionary 40 other than the true respondents. The learning efficiency of unnecessary words is greatly improved. Therefore, the learning in this case gives a negative reward in the same manner when the word is registered in the dictionary 40 of the person in charge other than the true respondent for the top word of the respondent mistakenly estimated by the system. , Reduce its weight.

FIG. 16 is a specific word learning process flow, and shows the specific word learning process executed by the reinforcement learning device 3 in step S178. The reinforcement learning device 3 reads the constants c3, L, S, min_idf, and max_ave_ratio from the definition file (step S211), and executes idf value re-evaluation processing (step S212). The idf value reevaluation process will be described later with reference to FIG. Next, the reinforcement learning device 3 sets the first matching word of the target non-responder in the mail analysis result table for the word t (step S213), and searches for t in the tf / idf dictionary 41 of the target non-responder. Tf · idf is set to the tf · idf value of the word matched with t (step S214), and max_tf · idf is set to the maximum tf · idf value of the target non-responder's tf · idf dictionary 41 (step S214). S215), it is checked whether or not tf · idf is greater than or equal to the product of max_tf · idf and max_ave_ratio (step S216). Note that max_ave_ratio represents an average weight standardized by the maximum values of tf · idf values and idf / conf values of all persons in charge, and the product of max_ave_ratio and max_tf · idf is an expression (14) described later. ) W _p ^d , and represents the weight effective for the score of the target non-respondent. A method of calculating max_ave_ratio will be described later (in FIG. 18). If f_tf · idf is not greater than or equal to the product of max_tf · idf and max_ave_ratio, the product of max_tf · idf and max_ave_ratio is set in f_tf · idf (step S217), and if greater than or equal to f_tf · idf A value obtained from tf · idf and c3 is set to idf (step S218).

  Thereafter, the reinforcement learning device 3 searches for t in the first word of the target non-respondent's idf / conf dictionary 42 and sets the idf / conf value of the word matched with t in idf / conf (step S219). , Max_idf / conf is set to the maximum idf / conf value in the idf / conf dictionary 42 of the target non-respondent (step S2110), and is idf / conf greater than or equal to the product of max_idf / conf and max_ave_ratio? Whether or not is checked (step S2111). Note that max_ave_ratio is as described above with reference to FIG. If f_idf / conf is not equal to or greater than the product of max_idf / conf and max_ave_ratio, the product of max_idf / conf and max_ave_ratio is set in f_idf / conf (step S2112), and if greater than or equal to the product, f_idf / A value obtained from idf / conf and c3 is set in conf (step S2113).

  Thereafter, the reinforcement learning device 3 updates the tf · idf value and idf / conf value of t in the tf · idf dictionary 41 and the idf / conf dictionary 42 (step S2114). That is, the new tf · idf value is the value obtained by adding f_tf · idf to the previous tf · idf value, the new idf / conf value is the value obtained by adding f_idf / conf to the previous idf / conf value And Thereafter, the reinforcement learning device 3 sets i = 2 (step S2115), and sets the i-th collation word of the target respondent in the mail analysis result table to the word t (step S2116). Next, the reinforcement learning device 3 searches for t in the target respondent's tf • idf dictionary 41 and sets tf • idf value of the word matched with t in tf • idf, and newly adds f_tf • idf and S F_tf · idf is calculated (step S2117), t is searched with the first word of the target respondent's idf / conf dictionary 42, idf / conf value of the word matched with t is set in idf / conf, and f_idf New f_idf / conf is calculated from / conf and S (step S2118), and the tf · idf value and idf / conf value of t in the tf · idf dictionary 41 and idf / conf dictionary 42 are updated (step S2119). That is, the new tf · idf value is the value obtained by adding f_tf · idf to the previous tf · idf value, the new idf / conf value is the value obtained by adding f_idf / conf to the previous idf / conf value And Thereafter, the reinforcement learning device 3 checks whether i is equal to L (step S2120). If it is not equal, i is set to i + 1 (step S2121), and step S2116 and subsequent steps are repeated. If equal, step S2121 is omitted and the process is terminated.

  FIG. 17 is an idf value re-evaluation process flow, and shows the idf value re-evaluation process executed by the reinforcement learning device 3 in step S212. The reinforcement learning device 3 calculates the idf value with the word set up to the top L rank of all non-responders in the mail analysis result table (step S221), and the words up to the top L rank of the target non-responder in the mail analysis result table Rewrite Score (d, t) to idf value (step S222), and delete the word with idf value less than min_idf of the target non-responder and words below L + 1 from the mail analysis result table (step S223) In the mail analysis result table, the words of the target non-responder are arranged in descending order by idf value (step S224). In step S223, if there is no word having an idf value less than min_idf or a word of L + 1 or lower, there may be no word to be deleted. Further, the maximum value of i may be smaller than L due to the deletion of the word in step S223. In this case, the maximum value of i is used instead of the read L. That is, if i is equal to the maximum value of i in step S2120, the process ends.

  A specific word is learned when a true respondent cannot be estimated as a respondent by the system. In this case, the weight of the important word may be smaller than the original value in the dictionary 40 of the true respondent. This can happen because the tf value of a synonym or similar word becomes extremely small when a true respondent uses a fixed number of technical terms and rarely uses the synonym or similar word. . Therefore, the higher word of the true respondent's score is compared with the upper word of another person in charge, and the word specificity is re-evaluated in the same manner as the idf value calculation method. In other words, if specificity is recognized in the upper word of a true respondent who was not estimated as an answerer, a positive reward is given as the specified word, and the weight is increased. The reinforcement method here is the same as that in the case where the true respondent in the equation (10) can be estimated except that the word sequence is determined by the idf value. However, the enhancement value of the word having the maximum idf value is the equation ( Equation (14) is used instead of 11).

Here, c3 is a constant as in equation (11). Note that the relationship between the standardized rank and weight for all the words in the dictionary 4 is a curve as shown in FIG. The horizontal axis of FIG. 18 represents a value obtained by dividing the rank of each word by the maximum rank of words of each person in charge (that is, the number of registered words), and the vertical axis similarly represents the weight of each word by the maximum weight value. Represents the divided value. FIG. 18 is a plot of the average weight (average of both tf · idf value and idf / conf value) of all persons in charge corresponding to each rank on the horizontal axis. Assuming that the average slope of this curve is “−1”, the value of the weight at the point of contact between this curve and the straight line with the slope of “−1” is max_ave_ratio in FIG. In equation (14), w _p ^d represents the product of this max_ave_ratio and the maximum weight. That is if the system the true respondent can not be estimated (classification) and a respondent, the higher word scores Score _t ^d of the true respondent is a low rank in the dictionary 40 of the personnel It is thought that there are many. The weight of such a low rank is a value that is many orders of magnitude smaller than the weight of a high rank. For this reason, even if the reinforcement value is given as a constant multiple of the original weight, it does not affect the overall score. Therefore, in response to the fact that the learning effect does not appear, if w ₁ ^d of the _first score is smaller than this w _p ^d , it is increased to w _p ^{d at a} time and strengthened. According to the study by the present inventor, if the average value of simple weights is w _p ^d , since there are many words with small weights, w _p ^d becomes a small value and the learning effect does not increase. understood.

  In this example, word matching at the time of learning is completely matched (therefore, words are always checked in FIGS. 13 to 17). When learning is used for learning, it is confirmed that the learning effect is improved without using partial matching as in mail analysis. When using mail 8 for learning, the word matching rate is set as the partial matching value. It may be multiplied and strengthened.

  As mentioned above, although this invention was demonstrated according to the embodiment, this invention can be variously deformed within the scope of the gist. For example, the document classification system of the present invention is not limited to the mail 8 and can be widely used for document classification. Further, the document to be classified can include a home page, for example. Furthermore, the document to be classified can include, for example, electronic data obtained by voice recognition of data input by voice. The document classification system of the present invention can classify electronic data configured in a natural language (including symbols). The document classification destination is not limited to each person in charge, and may be a department in charge in various organizations. The document classification system of the present invention may not include all three subsystems. That is, the dictionary editing device 1, the document classification device 2, and the reinforcement learning device 3 may be provided independently, or only the document classification device 2 may be provided, and the dictionary editing device 1 or the reinforcement learning device may be provided in the document classification device 2. 3 may be added.

  As described above, according to the present invention, in the dictionary editing apparatus, a tf / idf dictionary and an idf / conf dictionary are created by using tf / idf value and idf / conf value as two independent parameters. By using this as a dictionary for document classification devices, when used for classification of documents such as e-mails, practical classification accuracy can be obtained, and documents are automatically assigned to the appropriate person in charge. It is possible to easily create a dictionary for classifying correctly.

  Further, according to the present invention, in the document classification apparatus, when the above two dictionaries tf / idf dictionary and idf / conf dictionary are used for classification of documents such as e-mails, practical classification accuracy is obtained. And automatically and accurately classify documents to the appropriate personnel.

  Further, according to the present invention, the document classification system program is supplied by being stored in a medium such as a flexible disk, CD-ROM, CD-R / W, or DVD, or downloaded via a network such as the Internet. Therefore, the above-described document classification device can be easily realized, and accurate document classification can be performed.

It is a document classification system block diagram. It is a dictionary explanatory drawing. It is a document classification result (mail analysis result) explanatory drawing. It is a document classification processing flow. It is a dictionary edit processing flow. It is a person in charge / word table generation processing flow. It is a word combination table generation processing flow. It is word combination table explanatory drawing. It is an inquiry mail analysis processing flow. It is a tf / idf value weighted addition process flow. It is an idf / conf value weighted addition process flow. It is a reinforcement learning process flow. It is an important word learning process flow. It is an unnecessary word learning process flow. It is an unnecessary word reinforcement | strengthening process flow. It is a specific word learning process flow. It is an idf value re-evaluation processing flow. It is an explanatory view of the relationship between rank and weight.

Explanation of symbols

1 Dictionary editing device 2 Document classification device (mail analysis device)
3 Reinforcement learning device 4 Dictionary 40 Category dictionary 41 Importance dictionary (tf / idf dictionary)
42 Co-occurrence dictionary (idf / conf dictionary)

Claims (9)

A dictionary editing device for creating a dictionary based on a document in which a category belongs,
For each word that appears in the category, create an importance dictionary that stores tf and idf values that represent the importance of the word alone,
A dictionary editing apparatus that creates a co-occurrence dictionary that stores an idf / conf value representing co-occurrence between words for each combination of a plurality of words for words that appear in the category. The co-occurrence dictionary stores an idf / conf value representing co-occurrence between two words for each combination of a first word and a second word for a word that appears in the category. The dictionary editing device described. An importance dictionary that includes a plurality of category dictionaries provided for each of a plurality of categories, and each category dictionary stores a tf / idf value that represents the importance of a single word for each word that appears in the category, and A dictionary comprising a co-occurrence dictionary for storing an idf / conf value representing co-occurrence between two words for each combination of the first word and the second word for the appearing word;
Using the words that appear in the input document, the words are compared in the dictionary to obtain tf · idf values and idf / conf values for each word in the dictionary, and a predetermined calculation is performed based on these values. A score for each word is calculated, a score for each category is calculated based on the score for each word, and the input document is classified into one of the plurality of categories based on the score. Document classification device. 4. The document according to claim 3, wherein the document is an e-mail, the category is a person in charge of distributing the document, and the document classification device distributes the input document to the classified person in charge. Document classification device. 4. The document classification apparatus according to claim 3, wherein the document classification apparatus classifies the input document into a predetermined number of categories from the top in accordance with the score for each category. The document classification device calculates a score for each word by calculating a product of a tf · idf value for each word in the dictionary, an idf / conf value, and a matching rate between words appearing in the input document. The document classification device according to claim 3. The document classification device further includes:
Based on the result of classifying a predetermined document into one of the plurality of categories by the document classification device, the tf · idf value in the importance dictionary is updated, and the idf / conf value in the co-occurrence dictionary is updated. The document classification device according to claim 3, comprising: a learning device. The learning device
When the predetermined document is classified into a true category, a word that appears in the predetermined document and collated in the dictionary of the true category is regarded as an important word, and its weight is increased.
When the predetermined document is not classified into the true category, the specific word of the true category is re-evaluated by re-evaluating a word having high specificity with respect to the true category among the words appearing in the predetermined document. Increase the weight of
If the predetermined document is classified into a category other than the true category, the word that appears in the predetermined document and is matched with the dictionary of the category that has been incorrectly classified is used as an unnecessary word, and is erroneously classified. The document classification apparatus according to claim 7, wherein the weight in the dictionary of the selected category is reduced and the weight in the dictionary of a category other than the true category is reduced. A program for realizing a document classification device,
The program is stored in a computer.
Tf ・ idf that consists of a plurality of category dictionaries provided for each of a plurality of categories using words that appear in the input document, and each category dictionary represents the importance of each word for each word that appears in the category An importance dictionary for storing values, and a co-occurrence dictionary for storing idf / conf values representing co-occurrence between two words for each combination of the first word and the second word for words appearing in the category A process for collating the word with a dictionary to obtain a tf · idf value and an idf / conf value for each word in the dictionary;
A process of calculating a score for each word by performing a predetermined calculation based on the tf · idf value and the idf / conf value for each word;
Processing for calculating a score for each category based on the score for each word;
A document classification program for executing a process of classifying the input document into any of the plurality of categories based on a score for each category.

Images