JPWO2011052025A1 - Data processing apparatus, data processing method, and program - Google Patents

Abstract

The learning sample data extracting unit 140 extracts sample mails over a plurality of categories, and the learning sample selecting unit 150 is based on a learning sample selection rule 190 in which an upper limit value of the total number of sample mails and sample mail selection criteria are indicated. The total number of sample mails selected is maximized within the range of the upper limit value, and the number of selections that minimizes the difference between categories is calculated for each category, and categories are selected from the sample mails extracted by the learning sample data extraction unit 140. The sample mail used for learning of the learning unit 160 is selected for each category according to the number of units selected.

Description

  The present invention relates to a technique for classifying data into one of a plurality of categories.

Automatic classification by machine learning is often used as one of the methods for automatically classifying data into a plurality of categories.
Hereinafter, an automatic classification technique based on machine learning will be described using document data (hereinafter also simply referred to as a document) as an example.

In the automatic document classification method using machine learning, features of each category are learned using learning sample documents previously divided into a plurality of classification categories, and classification target documents are classified based on the learning results.
Therefore, the accuracy of document classification by machine learning depends on the learning sample document.
Patent Document 1 discloses a technique for improving classification accuracy by experimentally classifying learning sample documents, selecting and removing misclassified documents, and improving classification rules.

JP 2002-202984 A

However, the method as in Patent Document 1 has a problem that it takes time and labor to collect a large amount of correctly classified learning sample documents.
Also, in a system that applies document classification using machine learning, when there is a restriction on the time that can be used for machine learning and the user wants to determine the number of learning samples, in the method such as Patent Document 1, manual selection of learning samples is performed. There is a problem that work is required.

  Further, when the difference in the number of learning samples between classification categories is large, there is a problem that machine learning cannot be performed properly depending on the algorithm used, resulting in a decrease in classification accuracy.

In addition, when there are many documents with duplicate contents in the learning sample document, it is inefficient to learn all of these documents, and there is a problem that the time available for machine learning cannot be used effectively.
In order to generate classification rules with high classification accuracy within a limited time, it is desirable to learn as many documents with different contents and formats as possible. There is a problem that selection work is required.

  One of the main objects of the present invention is to solve the above-mentioned problems, and automatically selects sample data that can be efficiently learned without labor and effort. It is an object to provide a data classification technique that can improve classification accuracy by machine learning.

The data processing apparatus according to the present invention
A classification unit that classifies data into one of a plurality of categories according to a classification rule;
A learning unit that performs learning using sample data and newly generates a classification rule used by the classification unit;
A sample data extraction unit that extracts, for each category, sample data used for learning of the learning unit;
A sample data selection criterion information storage unit for storing sample data selection criterion information indicating an upper limit of the total number of sample data selections and a selection criterion for sample data;
Based on the sample data selection criteria information, the number of selected sample data is maximized within the range of the upper limit value, and the number of selections that minimizes the difference between categories is calculated for each category and extracted by the sample data extraction unit. And a sample data selection unit that selects sample data used for learning by the learning unit for each category according to the number of selections in category units.

  According to the present invention, the number of selections in which the total number of sample data selections is maximized within the range of the upper limit value and the difference between categories is minimized is calculated in category units, and the selection of category units is selected from the extracted sample data. Since the sample data used for learning is selected for each category according to the number, the classification accuracy in machine learning can be increased without manpower.

FIG. 3 is a diagram illustrating an example of a system configuration according to the first embodiment. 1 is a diagram illustrating a configuration example of a mail archive device according to Embodiment 1. FIG. FIG. 4 is a flowchart showing an operation example of the mail classification device according to the first embodiment. FIG. 4 is a flowchart showing an operation example of a learning sample selection unit according to the first embodiment. FIG. 4 is a flowchart showing a specific example of a sample mail selection operation according to the first embodiment. FIG. 4 is a flowchart showing a specific example of a sample mail selection operation according to the first embodiment. FIG. 4 is a diagram illustrating a configuration example of a document archive device according to a second embodiment. FIG. 3 is a diagram illustrating a hardware configuration example of a mail classification device and a document classification device according to the first and second embodiments.

  In the following description, an example in which electronic mail (hereinafter simply referred to as mail) is classified in the mail archiving system and an example in which document data is classified in the document archive system will be described. The present invention is not limited to the mail classification in the archive system and the document data classification in the document archive system, and can be applied to a general document classification system.

  In addition, in the following, by selecting the learning sample selection rules in advance considering the number of learning samples in each category and document information, selection of learning sample documents that enable efficient learning without manual labor A data processing technique that can automatically improve the classification accuracy by short-time machine learning will be described.

Embodiment 1 FIG.
FIG. 1 shows a configuration example of a mail archive system according to the present embodiment.
In FIG. 1, three organizations each including a user terminal and a mail server are shown.
Each organization has a domain that is an identifier of each organization.
For each user terminal included in each organization, an email address including the domain of the organization to which the user belongs is set.
Further, the mail server receives the mail and analyzes the domain included in the destination address of the received mail to determine the transfer destination of the received mail.

  In FIG. 1, for the sake of convenience, the mail domain is composed of a target organization domain and two non-target organization domains, but it is not necessary to be limited to the domain configuration of FIG. 1, and any domain configuration can be used. is there.

The target organization domain is a domain unique to the organization 301 in which the mail archive system of this embodiment is introduced.
As the organization 301, for example, an organization having a unique domain such as a company, a public organization such as a government office, a government office, other organizations, or an internal organization (business office, branch office, etc.) is applicable.
The two domains outside the target organization are unique domains of the organization 302 and the organization 303 that are not the organization 301, and are different domains.

In the configuration of FIG. 1, the organization 301 includes a mail server 311 and a user terminal 321.
The organization 302 includes a mail server 312 and a user terminal 322.
The organization 303 includes a mail server 313 and a user terminal 323.
The mail servers 311, 312, and 313 of each organization are connected through a network 330.
The user terminals 321, 322, and 323 can send and receive mail through the mail servers 311, 312, and 313 and the network 330.
Note that the number of user terminals and the configuration of the mail server are not limited to the configuration in FIG. 1, and an arbitrary number of user terminals and a mail server with an arbitrary configuration can be applied.

In the configuration of FIG. 1, the mail server 311 is connected to the mail archive device 200.
The mail archive device 200 includes a mail classification device 100.
The mail classification device 100 is an example of a data processing device.

  FIG. 2 shows a configuration example of the mail archive apparatus 200 according to the present embodiment.

The mail archive device 200 includes a mail storage database 210 and a mail classification device 100.
The mail archive device 200 duplicates the newly input mail 201 that is about to pass through the mail server 311 and stores it in the mail storage database 210.
Mail replication may be performed by the mail server 311.
The mail classification device 100 classifies the mail stored in the mail storage database 210 into a plurality of classification categories.
The classification results are accumulated in the classification result accumulation database 130.
The mail stored in the mail storage database 210 is given a mail ID for uniquely identifying the mail, and the mail stored in the mail storage database 210 and the classification result by this mail ID. The mail classification results stored in the storage database 130 are associated with each other.
The system administrator can query the mail storage database 210 and the classification result storage database 130 to refer to the mail classification result and perform mail search using the classification result as a key.
The mail classification device 100 is activated every period set by the system administrator, and the mail accumulated in the mail accumulation database 210 during the activation period is classified, so that all mails that are continuously input can be processed. It is possible to operate such that classification processing is performed.

The mail classification device 100 includes a classification target data extraction unit 110, a classification unit 120, a classification result storage database 130, a learning sample data extraction unit 140, a learning sample selection unit 150, a learning unit 160, a classification rule storage unit 170, and a condition rule. The storage unit 195 is configured.
The condition rule storage unit 195 stores learning sample extraction conditions 180 and learning sample selection rules 190 for a plurality of classification categories.
The learning sample selection rule 190 includes a sample total number upper limit determination rule 191, a sample number determination rule 192, and a sample selection rule 193.
There is no particular limitation on the number of classification categories, and an arbitrary natural number of 2 or more can be set.
The learning sample extraction condition 180 and the learning sample selection rule 190 are set by a system administrator or the like.

  The classification target data extraction unit 110 issues an inquiry sentence to the mail storage database 210 to the mail storage database 210, and extracts a mail corresponding to the inquiry sentence from the mail storage database 210 as a classification target mail.

  The classification unit 120 uses the classification rule generated by the learning unit 160 (stored in the classification rule storage unit 170) to convert the classification target email extracted by the classification target data extraction unit 110 into a plurality of classification categories. The data is classified into one of the classification categories, and the classification result is stored in the classification result accumulation database 130 in association with the mail ID.

The learning sample data extraction unit 140 is a mail that has been stored in the mail storage database 210 and that is a candidate for a learning sample to be used for learning by the learning unit 160 from among the classified mails (sample data). Is extracted for each category as a learning sample mail.
In the following, the learning sample mail is also referred to as a sample mail. On the other hand, the sample mail selected for learning by the learning unit 160 is referred to as a learning sample.
A mail extracted as a learning sample mail for a certain category category is a mail that matches the learning sample extraction condition 180 for the corresponding category category.
At this time, instead of newly taking out the learning sample mail from the mail storage database 210, the mail extracted from the mail storage database 210 is reduced by diverting the mail extracted by the classification target data extraction unit 110. can do.
The learning sample data extraction unit 140 is an example of a sample data extraction unit.

The learning sample extraction condition 180 can include, for example, a search expression using a regular expression.
By using regular expressions, it becomes possible to search for more complex patterns in addition to simple keywords, and the flexibility of the learning sample extraction condition 180 can be improved.

The learning sample extraction condition 180 used in the learning sample data extraction unit 140 may also extract and collate mail attributes.
A header field defined in Request For Comments (RFC) 2822, an envelope defined in RFC2821, or a header field uniquely defined for each mail server system is used.
Examples of header fields include From, To, Cc (sender, recipient, broadcast recipient address), Subject (subject), Date (transmission date / time), Received (reception date / time), and the like.

The learning sample extraction condition 180 used in the learning sample data extraction unit 140 may also extract the file name of the mail attachment file or the text in the attachment file and perform collation.
In the case of an attached file encoded in the format of Multipurpose Internet Mail Extension (MIME) defined in RFC2045-2049, the attached file name can be extracted from the MIME header, and the body is decoded and attached. It is possible to extract a file and extract text from the attached file.

  The learning sample selection unit 150 selects learning samples that are actually used for machine learning in the learning unit 160 in accordance with the learning sample selection rules 190.

The learning sample selection rule 190 includes three rules: a sample total number upper limit determination rule 191, a sample number determination rule 192, and a sample selection rule 193.
The sample total number upper limit determination rule 191 is a rule indicating the upper limit of the total number of learning samples (total number of selected sample data) of all classification categories used for machine learning.
The sample number determination rule 192 is a rule for determining the number of sample mails used for machine learning for each classification category based on the learning sample total number upper limit determined based on the sample total number upper limit determination rule 191. is there.
The sample selection rule 193 is a rule for selecting a sample mail for each classification category so that the number of samples for each classification category determined based on the sample number determination rule 192 is obtained.
The sample total number upper limit determination rule 191, the sample number determination rule 192, and the sample selection rule 193 show the upper limit of the total number of selected learning samples and the sample mail selection criteria as described above. It corresponds to the example.
Further, the condition rule storage unit 195 stores these rules corresponding to the example of the sample data selection reference information, and is an example of the sample data selection reference information storage unit.

Then, based on the learning sample selection rule 190, the learning sample selection unit 150 calculates the number of selections in which the total number of sample mail selections is maximum within the range of the upper limit value and the difference between the categories is minimum, and learning is performed. A learning sample to be used for learning by the learning unit 160 is selected for each category from the sample mail extracted by the sample data extraction unit 140 according to the number of categories selected.
As described above, the learning sample selection unit 150 calculates the number of selections in which the total number of selected sample mails is maximum within the range of the upper limit value and the difference between the categories is minimum. “Minimum” means that when the number of categories is two, the difference between the two categories is minimum.
When the number of categories is three or more, it means that the difference between the category with the largest selection number and the category with the smallest selection number is the smallest.
The learning sample selection unit 150 is an example of a sample data selection unit.

  The learning unit 160 inputs the mail extracted and selected for each category by the learning sample data extraction unit 140 and the learning sample selection unit 150 as a learning sample of each category, and uses the input learning sample to classify A classification rule used for classification of the unit 120 is generated.

The classification unit 120 and the learning unit 160 can use a generally known document classification method using machine learning.
A document classification method using a plurality of machine learnings can also be used.

  Next, the operation of the mail classification device 100 will be described with reference to FIG.

The mail classification device 100 is activated in accordance with the activation cycle preset by the system administrator or the like (S101).
The classification target data extraction unit 110 extracts mail stored in the mail storage database 210 during one cycle from the previous activation of the mail classification apparatus 100 to the current activation as classification target mail (S102).
The classification unit 120 classifies the mail extracted as the classification target mail into one of the classification categories according to the classification rule, and accumulates the classification result in the classification result accumulation database 130 (S103) (classification processing).
The learning sample data extraction unit 140 extracts the learning sample mail using the learning sample extraction condition 180 set for each classification category (S104) (sample data extraction processing).
The learning sample selection unit 150 reads out the learning sample selection rule 190 from the condition rule storage unit 195 (reading process), and selects learning samples to be actually used for machine learning from the learning sample mail for each classification category ( S105) (sample data selection process).
The learning unit 160 learns learning samples extracted and selected by classification category, and generates or updates a classification rule (S106) (learning process).
The above series of operations is repeated every time the mail classification device 100 is activated.

  The operation (S105) of the learning sample selection unit 150 will be described in more detail with reference to FIG.

First, the learning sample selection unit 150 determines the upper limit value of the total number of learning samples in all classification categories according to the sample total number upper limit determination rule 191 (S201).
Next, the learning sample selection unit 150 determines the number of sample mails for each classification category used for machine learning based on the learning sample total number upper limit determined in S201 according to the sample number determination rule 192 (S202).
Finally, the learning sample selection unit 150 determines a sample mail to be selected for each classification category so that the number of sample mails for each classification category determined in S202 is obtained according to the sample selection rule 193 (S203).

  The sample total number upper limit determination rule 191 is determined by a system administrator or the like in consideration of the time that the mail classification device 100 can allocate to machine learning. Specific examples are given below.

If the time that the mail classification device 100 assigns to one machine learning is set as a fixed time, the number of mails that can be machine-learned can be calculated from the average learning processing time per mail and the total time that can be assigned to machine learning. .
Therefore, by giving the number of mails calculated here as a constant as the upper limit of the total number of learning samples, it is guaranteed that machine learning is always completed in time.
Since the mail archiving apparatus 200 is activated at a predetermined activation cycle, the number of newly input mails 201 may not be equal to the constant given as the learning sample total number upper limit depending on the activation time.
In this case, the number of new input emails 201 may be given again as the upper limit value of the total number of learning samples.

Further, the upper limit value of the total number of learning samples may be determined as a ratio to the number of new input emails 201.
In other words, the sample total number upper limit determination rule 191 defines a rule in which a value obtained by multiplying the number of classification target emails input in each cycle by a predetermined ratio is a sample total number upper limit, and the learning sample selection unit 150 Based on the sample total number upper limit determination rule 191, a value obtained by multiplying the number of inputted classification target mails by a predetermined ratio is calculated as the sample total number upper limit for each cycle, and the calculated sample total number upper limit is calculated. You may make it select a learning sample using.
In this way, the upper limit value of the total number of learning samples is not given as a constant, but can be changed according to the number of new input mails 201.
The number of new input mails 201 input to the mail classification device 100 varies depending on the activation cycle, but when the total number of mails input throughout the day is fixed to some extent, the upper limit of the total number of learning samples is set as described above. By setting, the time allotted to machine learning by the system throughout the day becomes a substantially constant value.
In addition, by setting the upper limit value of the total number of learning samples in this way, it becomes possible to select learning samples from the mails input to the system throughout the day without any bias.

The sample number determination rule 192 is determined by a system administrator or the like according to the characteristics of the system. Specific examples are given below.
In the following description, an example in which there are two classification categories, category A and category B, is shown, but the same rule can be given when the number of categories is three or more.

The number of category A mails extracted by the learning sample data extraction unit 140 is a, and the number of category B mails is b.
In addition, the upper limit according to the sample total number upper limit determination rule 191 is c (≦ a + b).
At this time, a sample number determination rule 192 indicating the following criteria is provided, and the learning sample selection unit 150 calculates the selection number a ′ of the category A sample mail and the selection number b ′ of the category B sample mail based on the following criteria. To do.

1) When a <c / 2,
a '= a
b ′ = c−a
2) When b <c / 2,
a '= b
b ′ = c−b
3) In cases other than 1) and 2) above,
a ′ = c / 2
b ′ = c / 2

By doing so, the difference between the number of learning samples in category A (a ′) and the number of learning samples in category B (b ′) is minimized under the condition that the total number of learning samples is the upper limit c. You can set the number of sample emails.
In general, it is known that the accuracy of document classification using machine learning increases as the difference in the number of learning samples for each classification category increases. Therefore, the accuracy of mail classification can be increased by the above rules.

As described above, when the learning sample total number upper limit value is a constant, the number of new input emails 201 (a + b) may not be equal to the constant given as the learning sample total number upper limit value. As described above, the upper limit (c) of the total number of learning samples is changed to the number (a + b) of newly input mail 201 (c = a + b).
Further, when the upper limit value of the total number of learning samples is set as a ratio to the number of new input mails 201, c ≦ a + b is always satisfied.
Therefore, the relationship between a, b, and c is always c ≦ a + b, and the above rule can be applied.
Thus, when the total number of sample emails extracted by the learning sample data extraction unit 140 is less than the upper limit (constant) of the total number of sample emails, the total number of sample emails is the total number of sample emails extracted. The number of selections that maximizes the range and minimizes the difference between categories is calculated for each category.

  In addition, a sample number determination rule 192 indicating the following criteria is provided, and the learning sample selection unit 150 calculates the selection number a ′ of the category A sample mail and the selection number b ′ of the category B sample mail based on the following criteria. You may do it.

1) When a <c / 2,
a '= a
b '= a
2) When b <c / 2 a ′ = b
b '= b
3) In cases other than 1) and 2) above,
a ′ = c / 2
b ′ = c / 2

This is a rule such that the total number of learning samples is closest to the upper limit value c under the condition that the number of learning samples in category A is always equal to the number of learning samples in category B.
This rule is effective when both category A and category B learning samples are sufficiently available.

  In addition, a sample number determination rule 192 indicating the following criteria is provided, and the learning sample selection unit 150 calculates the selection number a ′ of the category A sample mail and the selection number b ′ of the category B sample mail based on the following criteria. You may do it.

a ′ = (a * c) / (a + b)
b ′ = (b * c) / (a + b)

  This is a rule that maintains the ratio of the original number of learning samples a and b, and that the total number of learning samples is equal to the upper limit value c.

The sample selection rule 193 is determined by the system administrator or the like according to the characteristics of the system.
For example, from among the emails extracted by the learning sample data extraction unit 140, randomly select a sample email for each classification category so as to be the number of samples for each classification category determined by the sample count determination rule 192, or When the transmission date / time information is provided in the sample mail, it is possible to define a rule such that the date / time indicated in the transmission date / time information is selected in order from the newest.

Preferably, in each classification category, the number of times sample mail having the same attribute (for example, mail subject (title)) is selected is minimized, and the distribution of the number of sample mails having each attribute is minimized. In addition, a sample mail may be selected under the condition.
In other words, the learning sample data extraction unit 140 extracts a plurality of sample emails having a plurality of types of attributes (email subject (title)) for one category, and the learning sample selection unit 150 selects each category. The learning sample mail is selected so that the number of sample mail attribute types is maximized and the difference in the number of sample mail selections between attributes is minimized within the specified number of selections.
Here, “the difference in the number of selected sample mails between attributes is minimum” means that the difference between the two categories is minimum when the number of attribute types is two.
When the number of attribute types is three or more, it means that the difference between the attribute with the largest selection number and the attribute with the smallest selection number is the smallest.
A specific operation example in the case where the attribute example is the mail subject will be described with reference to FIG.

FIG. 5 illustrates an operation of selecting N mails from a certain category.
The mail group extracted as a sample of this category by the learning sample data extraction unit 140 is represented by M.
First, the learning sample selection unit 150 selects the maximum number of emails so as not to select emails with the same subject more than once (S301).
If the number of selected mails is N or more (S302), the learning sample selection unit 150 randomly selects mails so as to be N mails and ends (S303).
If the number of selected emails is smaller than N (S302), the learning sample selection unit 150 determines that the selected email has been selected (S304), excludes the number of confirmed emails from M (S305), and proceeds to S301. Return.
And the learning sample selection part 150 repeats a process until the number of confirmed mail becomes N.

As described above, the learning sample selection unit 150 selects one sample mail for each attribute for all attributes (subjects) from a plurality of sample mails for each category, and the number of selected sample mails is included in the category. If it is equal to or greater than the designated selection number (N in the above example), sample data used for learning is randomly selected from the selected sample mail so as to match the selection number (S303).
On the other hand, if the number of selected sample emails is less than the number of selections specified in the category, the learning sample selection unit 150 selects the selected sample emails as sample emails used for learning (S304), and unselected One sample mail is selected for each attribute for all the attributes included in the unselected sample mail from the sample mails, and a shortage of sample mails is selected.

  Since emails with the same subject are likely to have very similar contents, the above-mentioned selection rule eliminates duplicate learning samples, resulting in a wide variety of topics selected as learning samples. It can be included.

Alternatively, the extension of the attached file of the sample mail may be used as an example of the attribute.
In other words, the sample mail attachments to be selected have the largest number of extension types (application program types), and the sample mails so that the distribution of the number of sample mails with files with each extension attached is minimized. May be selected.
Since files with the same extension are likely to have similar document formats, emails selected as learning samples by the selection rules as described above can include attachments in various formats. Become.

Alternatively, when the selected sample mail is a received mail, the domain of the mail address described in the From field of the header field may be used as an example of the attribute.
In other words, even if the sample mail is selected so that the number of domain types of the mail address described in the From field of the header field is the largest and the distribution of the number of sample mails including the mail addresses having each domain is minimized. Good.
When the selected sample mail is a transmission mail, the domain of the mail address described in the header field To may be used as an example of the attribute.
In other words, even if the sample mail is selected so that the domain type of the mail address described in the header field To is the largest, and the distribution of the number of sample mails including the mail address having each domain is minimized. Good.
E-mails with the same domain are likely to be similar in content, so it is possible to include a wide variety of topics in e-mails selected as learning samples by the above selection rules. .

Alternatively, the character code type described in charset included in the header field Content-type of the sample mail may be used as an example of the attribute.
In other words, the sample mail is set so that the number of character codes described in the charset included in the header field Content-type of the selected sample mail is the largest, and the distribution of the number of sample mails in which each character code is written is minimized. You may choose.
Since emails with the same character code are likely to have been created in the same language area, emails selected as learning samples by the selection rules as described above can include various languages. Become.

The sample selection rules listed above may be set in combination in consideration of the priority order.
For example, the operation when priority is given to the selection of sample mails having various subjects, secondly to the selection of sample mails having various domains, and thirdly, various character codes will be described with reference to FIG. .

First, the learning sample selection unit 150 repeats the operations of S301, S302, S304, and S305 of FIG. 5 and lists sample mail selection candidates that include various subject mails (S401).
When there is one candidate (S402), the learning sample selection unit 150 determines the selection mail and ends.
When there are a plurality of candidates (S402), the learning sample selection unit 150 narrows down the candidates so as to include various domains from the candidates (S403).
Furthermore, when there are a plurality of candidates (S404), the learning sample selection unit 150 narrows down the candidates to include various character codes (S405).
Further, when there are a plurality of candidates (S406), the learning sample selection unit 150 randomly selects one candidate and ends (S407).

More preferably, a mail whose text size is smaller than a value predetermined by a system administrator or the like may not be selected as a learning sample.
For example, when it is desired not to select a mail whose number of characters included in the text is 10 or less, if the text is encoded by UTF-8 (8-bit UCS Transformation Format), it is assumed that the average of 3 bytes per character is 30. You may decide not to select emails that are less than bytes.
Since a text file with a small size does not have much feature quantity, machine learning cannot be performed properly depending on the algorithm used, and as a result, the classification accuracy may be lowered.
Therefore, it is possible to improve the classification accuracy by excluding small mails from the learning sample.

  As described above, in the first embodiment, the learning sample to be used in the mail classification device 100 is automatically selected according to the preset learning sample selection rule 190, so that labor for creating the learning sample manually is reduced. It is possible to select a learning sample so that efficient learning can be performed without applying a mark, and as a result, it is possible to provide a document classification device that improves classification accuracy by short-time machine learning.

Embodiment 2. FIG.
FIG. 7 is a configuration diagram showing a document archive system to which the document classification apparatus 100b according to the present embodiment is applied.
The difference between the document archive device 200b and the mail archive device 200 is that the input is a document file or an electronic mail.
That is, as shown in FIG. 7, the configuration of the document classification device 100b and the document archive device 200b is substantially the same as that of the mail classification device 100 and the mail archive device 200 shown in FIG.
Hereinafter, the operation of the document archive device 200b will be described focusing on differences from the mail archive device 200.

The document archive device 200b includes a document storage database 210b and a document classification device 100b.
The document archive device 200b duplicates the new input document 201b and stores it in the document storage database 210b.
The document stored in the document storage database 210b is assigned a document ID for uniquely identifying the document, and the document stored in the document storage database 210b by this document ID and the classification result storage. The document classification results stored in the database 130 are associated with each other.
The system administrator makes an inquiry to the document storage database 210b and the classification result storage database 130, thereby making it possible to refer to the document classification result and perform a document search using the classification result as a key.
The document classification apparatus 100b is activated every period set by the system administrator, and the documents accumulated in the document accumulation database 210b during the activation period are classified, so that all documents input continuously It is possible to operate such that classification processing is performed.

The document storage database 210b may hold additional information such as a document file name associated with the stored document, its extension, document creation date and time, creator, and the like.
This makes it possible to use these additional information in the learning sample selection rule 190.

  Below, the setting method of the sample selection rule 193 among the learning sample selection rules 190 is demonstrated. Other rules are the same as in the mail archive system.

The sample selection rule 193 is determined by the system administrator or the like according to the characteristics of the system.
For example, a sample document for each classification category is randomly selected from the documents extracted by the learning sample data extraction unit 140 so that the number of samples for each classification category determined by the sample number determination rule 192 is obtained, or When the document creation date / time information is provided in the sample document, it is possible to define a rule such that the date / time indicated in the document creation date / time information is selected in order from the newest.

Further, it is conceivable to select a sample document based on the attribute of the sample document as in the mail archive system.
Specifically, the file name (title) of the sample document is an example of the attribute, the number of times the sample document having the same file name is selected in each classification category is minimized, and the number of sample documents of each file name is A sample document may be selected under the condition that the variance is minimized.
Since documents with the same file name are likely to have very similar contents, the learning samples with duplicate contents are eliminated by the selection rules as described above, and as a result, various topics are selected as the learning samples. Can be included.

Alternatively, the sample documents may be selected so that the number of types of extension (types of application programs) of the selected sample document is the largest and the distribution of the number of sample documents having each extension is minimized.
Since files with the same extension are likely to have similar document formats, it is possible to include documents of various formats that are selected as learning samples by the above selection rules. .

Alternatively, the sample documents may be selected so that the number of creators of sample documents to be selected is the largest and the variance of the number of sample documents created by each creator is minimized.
Since documents created by the same person are likely to have similar topics, documents selected as learning samples by the above selection rules can include a wide variety of topics.

The sample document selection rules listed above may be set in combination in consideration of the priority order.
For example, it is possible to set rules such that selection of sample documents having various file names has the highest priority, and secondly selection of sample documents having various extensions.

  As described above, in the second embodiment, the learning sample to be used in the document classification device 100b is automatically selected according to a preset learning sample selection rule 190, so that labor for creating the learning sample manually is reduced. It is possible to select a learning sample so that efficient learning can be performed without applying a mark, and as a result, it is possible to provide a document classification device that improves classification accuracy by short-time machine learning.

As described above, in Embodiments 1 and 2, the document classification apparatus classifies the input document into a plurality of classification categories.
Means for inputting a plurality of sample documents;
Means for extracting a sample document matching the sample extraction condition as a sample document of the corresponding classification category according to the sample extraction condition set in advance for each classification category;
Means for selecting a learning sample document to be used in each classification category from the extracted sample document according to a learning sample selection rule set in advance;
Means for generating or updating a classification rule by performing machine learning using at least one algorithm using a learning sample document selected for each classification category;
Means for inputting one or more documents to be classified;
Means for classifying the input classification target document into a plurality of classification categories using the classification rule;
A document classification apparatus having means for outputting a classification result of a classification target document has been described.

In the first and second embodiments,
The learning sample selection rule is:
A sample total number upper limit determination rule that determines the upper limit of the total number of learning samples for all classification categories used for machine learning,
Based on the learning sample total number upper limit value determined by the sample total number upper limit determination rule, the sample number determination rule that determines the number of samples for each classification category used for machine learning,
It consists of three rules, the sample selection rule that selects sample documents for each classification category, so that the number of samples for each classification category determined by the sample number determination rule,
The means for selecting the learning sample document is:
A method for determining the upper limit for the total number of learning samples for all classification categories used for machine learning according to the rule for determining the upper limit for the total number of samples,
A method for determining the number of samples for each classification category to be used for machine learning according to the sample number determination rule,
A means for selecting sample documents extracted for each classification category according to sample selection rules;
Explained that it consists of.

In the first and second embodiments,
The sample total number upper limit determination rule is:
Specify the upper limit of the total number of learning samples for all classification categories used for machine learning as a constant,
A rule that specifies the learning sample total number upper limit value as the input sample number only when the number of samples input in the means for inputting the plurality of sample documents is less than the constant specified as the learning sample total number upper limit value. Explained that there is.

In the first and second embodiments,
The sample total number upper limit determination rule is:
Rule for determining the upper limit of the total number of learning samples of all classification categories used for machine learning as a value obtained by multiplying the number of samples input by the means for inputting a plurality of sample documents by a predetermined sample document usage rate I explained that.

In the first and second embodiments,
The sample number determination rule is:
Under the condition that the total number of samples in each classification category is equal to the upper limit of the total number of learning samples, the difference between the number of samples in the category with the most sample documents and the category with the least number of samples is minimized, and It has been explained that the rule determines the number of samples in each classification category so that the variance of the number of samples in each classification category is minimized.

In the first and second embodiments,
The sample number determination rule is:
A rule that determines the number of samples for each classification category so that the total number of samples for each classification category is closest to the upper limit for the total number of learning samples under the condition that the number of samples for each classification category is all equal. Explained that there is.

In the first and second embodiments,
The sample number determination rule is:
Under the condition that the total number of samples of each classification category is equal to the upper limit of the total number of learning samples, the ratio of the number of sample documents of each classification category input in the means for inputting the plurality of sample documents; It has been explained that the rule is to determine the number of samples of each classification category so that the ratio of the number of sample documents extracted as sample documents of each classification category becomes equal.

In the first and second embodiments,
It has been explained that the sample selection rule is a rule for randomly selecting sample documents of each classification category.

In the first and second embodiments,
The document classification device has information regarding the creation date and time of the input sample document,
It has been explained that the sample selection rule is a rule for selecting a sample document in order of creation date and time.

In the first embodiment,
The sample document and the classification target document are emails,
It has been explained that the sample selection rule is a rule for selecting a sample e-mail in order of transmission date and time.

In the second embodiment,
The document classification device has information about the document file name of the input sample document,
The sample selection rule selects the sample document so that the number of times the sample document having the same document file name is selected in each classification category is minimized and the distribution of the number of samples having each document file name is minimized. Explained that it is a rule to do.

In the first embodiment,
The sample document and the classification target document are emails,
The sample selection rules are:
It is a rule to select sample emails that minimize the number of times sample emails with the same subject are selected for each category and minimize the variance of the number of sample emails with each subject. explained.

In the second embodiment,
The document classification device has application information indicating the type of created application for the input sample document,
The sample selection rule is a rule for selecting a sample document so that the number of types of application information held by the sample document selected in each classification category is the largest and the variance of the number of samples having each application information is minimized. Explained that there is.

In the first embodiment,
The sample document and the classification target document are emails,
According to the sample selection rule, in each classification category, the type of application information included in the selected sample email attachment is the largest, and the distribution of the number of sample emails attached with the file having each application information is minimized. I explained that it is a rule to select sample e-mails.

In the second embodiment,
The document classification device has information indicating the creator of the input sample document,
The sample selection rule is a rule for selecting a sample document so that the number of creators of sample documents selected in each classification category is the largest and the variance of the number of samples created by each creator is minimized. Explained that there is.

In the first embodiment,
The sample document and the classification target document are emails,
The sample selection rule minimizes the number of times that a sample email having the same domain is selected for the email address described in a specific header field of the email in each classification category, and the sample electronic having each domain. We explained that it is a rule to select sample emails so that the distribution of the number of emails is minimized.

In the first embodiment,
The sample document and the classification target document are emails,
The sample selection rule minimizes the number of times sample emails created with the same character code are selected in each classification category and minimizes the variance of the number of sample emails created with the same character code. Explained that it is a rule to select sample emails.

In the first and second embodiments,
It has been explained that the sample selection rule is a rule for not selecting a document whose size is less than or equal to a predetermined value as a sample document.

Finally, a hardware configuration example of the mail classification device 100 and the document classification device 100b described in the first and second embodiments will be described.
FIG. 8 is a diagram illustrating an example of hardware resources of the mail classification device 100 and the document classification device 100b described in the first and second embodiments.
The configuration in FIG. 8 is merely an example of the hardware configuration of the mail classification device 100 and the document classification device 100b, and the hardware configuration of the mail classification device 100 and the document classification device 100b is the configuration described in FIG. The configuration is not limited to this, and other configurations may be used.

In FIG. 8, the mail classification device 100 and the document classification device 100b include a CPU 911 (also referred to as a central processing unit, a central processing unit, a processing unit, an arithmetic unit, a microprocessor, a microcomputer, and a processor) that executes a program.
The CPU 911 is connected to, for example, a ROM (Read Only Memory) 913, a RAM (Random Access Memory) 914, a communication board 915, a display device 901, a keyboard 902, a mouse 903, and a magnetic disk device 920 via a bus 912. Control hardware devices.
Further, the CPU 911 may be connected to an FDD 904 (Flexible Disk Drive), a compact disk device 905 (CDD), a printer device 906, and a scanner device 907. Further, instead of the magnetic disk device 920, a storage device such as an optical disk device or a memory card (registered trademark) read / write device may be used.
The RAM 914 is an example of a volatile memory. The storage media of the ROM 913, the FDD 904, the CDD 905, and the magnetic disk device 920 are an example of a nonvolatile memory. These are examples of the storage device.
The “classification result storage database 130”, “classification rule storage unit 170”, and “condition rule storage unit 195” described in the first and second embodiments are realized by the RAM 914, the magnetic disk device 920, and the like.
A communication board 915, a keyboard 902, a mouse 903, a scanner device 907, an FDD 904, and the like are examples of input devices.
The communication board 915, the display device 901, the printer device 906, and the like are examples of output devices.

  As shown in FIG. 1, the communication board 915 is connected to, for example, a mail server. The communication board 915 may be connected to, for example, a LAN (local area network), the Internet, a WAN (wide area network), a SAN (storage area network), or the like.

The magnetic disk device 920 stores an operating system 921 (OS), a window system 922, a program group 923, and a file group 924.
The programs in the program group 923 are executed by the CPU 911 using the operating system 921 and the window system 922.

The RAM 914 temporarily stores at least part of the operating system 921 program and application programs to be executed by the CPU 911.
The RAM 914 stores various data necessary for processing by the CPU 911.

The ROM 913 stores a BIOS (Basic Input Output System) program, and the magnetic disk device 920 stores a boot program.
When the mail classification device 100 and the document classification device 100b are activated, the BIOS program in the ROM 913 and the boot program in the magnetic disk device 920 are executed, and the operating system 921 is activated by the BIOS program and the boot program.

  The program group 923 executes the function described as “˜part” in the description of the first and second embodiments (the same applies to the following, except for “classification rule storage unit 170” and “condition rule storage unit 195”). Program to be stored. The program is read and executed by the CPU 911.

The file group 924 includes “selection of”, “selection of”, “extraction of”, “determination of”, “determination of”, “learning of” in the description of the first and second embodiments. ”,“ Comparison of ”,“ Generation of ”,“ Update of ”,“ Setting of ”,“ Registration of ”, etc. Values and parameters are stored as items of “˜file” and “˜database”.
The “˜file” and “˜database” are stored in a recording medium such as a disk or a memory. Information, data, signal values, variable values, and parameters stored in a storage medium such as a disk or memory are read out to the main memory or cache memory by the CPU 911 via a read / write circuit, and extracted, searched, referenced, compared, and calculated. Used for CPU operations such as calculation, processing, editing, output, printing, and display.
Information, data, signal values, variable values, and parameters are stored in the main memory, registers, cache memory, and buffers during the CPU operations of extraction, search, reference, comparison, calculation, processing, editing, output, printing, and display. It is temporarily stored in a memory or the like.
In addition, the arrows in the flowcharts described in the first and second embodiments mainly indicate input / output of data and signals, and the data and signal values are the RAM 914 memory, the FDD 904 flexible disk, the CDD 905 compact disk, and the magnetic field. Recording is performed on a recording medium such as a magnetic disk of the disk device 920, other optical disks, mini disks, DVDs, and the like. Data and signals are transmitted online via a bus 912, signal lines, cables, or other transmission media.

In addition, what is described as “˜unit” in the description of the first and second embodiments may be “˜circuit”, “˜device”, “˜device”, and “˜step”, It may be “˜procedure” or “˜processing”.
That is, the data processing method according to the present invention can be realized by the steps, procedures, and processes shown in the flowcharts described in the first and second embodiments.
Further, what is described as “˜unit” may be realized by firmware stored in the ROM 913. Alternatively, it may be implemented only by software, or only by hardware such as elements, devices, substrates, and wirings, by a combination of software and hardware, or by a combination of firmware. Firmware and software are stored as programs in a recording medium such as a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, and a DVD. The program is read by the CPU 911 and executed by the CPU 911. That is, the program causes the computer to function as “to part” in the first and second embodiments. Alternatively, the computer executes the procedure and method of “to unit” in the first and second embodiments.

  As described above, the mail classification device 100 and the document classification device 100b according to the first and second embodiments are configured such that a CPU as a processing device, a memory as a storage device, a magnetic disk, etc., a keyboard as an input device, a mouse, a communication board, etc. The computer includes a display device, a communication board, and the like, and implements the functions indicated as “˜unit” using the processing device, the storage device, the input device, and the output device as described above.

  100 mail classification device, 100b document classification device, 110 classification target data extraction unit, 120 classification unit, 130 classification result accumulation database, 140 learning sample data extraction unit, 150 learning sample selection unit, 160 learning unit, 170 classification rule storage Part, 180 learning sample extraction condition, 190 learning sample selection rule, 191 sample total number upper limit determination rule, 192 sample number determination rule, 193 sample selection rule, 195 condition rule storage unit, 200 mail archive device, 200b document archive device, 201 New Input Mail, 201b New Input Document, 210 Mail Storage Database, 210b Document Storage Database, 301 Organization, 302 Organization, 303 Organization, 311 Mail Server, 312 Mail Server, 313 Mail server, 321 user terminal, 322 user terminal, 323 user terminal, 330 network.

Claims (17)

A classification unit that classifies data into one of a plurality of categories according to a classification rule;
A learning unit that performs learning using sample data and newly generates a classification rule used by the classification unit;
A sample data extraction unit that extracts, for each category, sample data used for learning of the learning unit;
A sample data selection criterion information storage unit for storing sample data selection criterion information indicating an upper limit of the total number of sample data selections and a selection criterion for sample data;
Based on the sample data selection criteria information, the number of selected sample data is maximized within the range of the upper limit value, and the number of selections that minimizes the difference between categories is calculated for each category and extracted by the sample data extraction unit. A data processing apparatus comprising: a sample data selection unit that selects sample data to be used for learning by the learning unit for each category according to the number of selections in category units from the sample data. The sample data selection criteria information storage unit is
Stores sample data selection criteria information indicating the selection criteria of matching the total number of sample data selections to the upper limit and minimizing the difference in the number of selections between categories,
The sample data selection unit
2. The data processing according to claim 1, wherein, based on the sample data selection criterion information, the number of selections in which the total number of sample data matches the upper limit value and the difference between categories is minimized is calculated for each category. apparatus. The sample data selection unit
When the number of sample data of category A is a, the number of sample data of category B is b, the upper limit of the total number of sample data selection is c, and c ≦ a + b,
1) When a <c / 2,
a '= a
b ′ = c−a
2) When b <c / 2,
a '= b
b ′ = c−b
3) In cases other than 1) and 2) above,
a ′ = c / 2
b ′ = c / 2
3. The data processing apparatus according to claim 2, wherein the selection number a ′ of category A and the selection number b ′ of category B are calculated by the following. The sample data selection criteria information storage unit is
Stores sample data selection criteria information indicating the selection criteria that the total number of selected sample data is the maximum within the range of the upper limit and the number of selections is the same in all categories,
The sample data selection unit
2. The data processing apparatus according to claim 1, wherein, based on the sample data selection criterion information, the number of selected sample data is maximized within the range of the upper limit value, and the number of selections is the same in all categories. The sample data selection unit
When the number of sample data of category A is a, the number of sample data of category B is b, the upper limit of the total number of sample data selection is c, and c ≦ a + b,
1) When a <c / 2,
a '= a
b '= a
2) When b <c / 2 a ′ = b
b '= b
3) In cases other than 1) and 2) above,
a ′ = c / 2
b ′ = c / 2
5. The data processing apparatus according to claim 4, wherein the selection number a ′ of category A and the selection number b ′ of category B are calculated by the following. The sample data selection unit
When the total number of sample data extracted by the sample data extraction unit is less than the upper limit of the total number of sample data selected, the total number of sample data selected is the maximum within the range of the total number of sample data extracted, The data processing apparatus according to claim 1, wherein the selection number that minimizes the difference is calculated for each category. The sample data extraction unit
Extract multiple sample data with multiple attributes for one category,
The sample data selection unit
For each category, sample data used for learning is selected so that the number of sample data attribute types is maximized and the difference in the number of sample data selections between attributes is minimized within the specified number of selections. The data processing apparatus according to claim 1. The sample data selection unit
For each category, select one sample data for each attribute for all the attributes included in the plurality of sample data from a plurality of sample data of the corresponding category,
If the number of selected sample data is greater than or equal to the number of selections specified in the category, select sample data to be used for learning at random so that it matches the number of selections from the selected sample data,
If the number of selected sample data is less than the selected number specified in the category, the selected sample data is selected as sample data used for learning, and the unselected sample data is selected from the unselected sample data. 8. The data processing apparatus according to claim 7, wherein one sample data is selected for each attribute of all the included attributes, and insufficient sample data is selected. The sample data extraction unit
Extract multiple sample data each with date and time information set,
The sample data selection unit
9. The data processing apparatus according to claim 8, wherein when there are a plurality of sample data having the same attribute, the sample data is selected in the order of date and time indicated in the date and time information. The sample data extraction unit
Extract multiple sample data showing multiple titles for one category,
The data processing device includes:
The sample data selection unit
For each category, the sample data used for learning is selected so that the number of types of titles is maximized and the difference in the number of sample data selected between titles is minimized within the range of the designated number of selections. Item 8. The data processing device according to Item 7. The sample data extraction unit
For one category, extract multiple sample data that shows multiple types of data creators,
The sample data selection unit
For each category, select the sample data to be used for learning so that the number of types of data creators is maximized and the difference in the number of sample data choices between data creators is minimized within the specified number of selections. 8. The data processing apparatus according to claim 7, wherein The sample data extraction unit
For a single category, extract multiple emails with multiple email addresses belonging to multiple domains as multiple sample data,
The sample data selection unit
For each category, the sample data used for learning is selected so that the number of types of domains is maximized and the difference in the number of sample data selected between domains is minimized within the range of the specified number of selections. Item 8. The data processing device according to Item 7. The sample data extraction unit
For one category, extract multiple sample data associated with multiple types of application programs,
The sample data selection unit
For each category, sample data used for learning is selected so that the number of types of application programs is maximized and the difference in the number of sample data selected between application programs is minimized within the range of the specified number of selections. The data processing apparatus according to claim 7. The sample data extraction unit
Extract multiple sample data created with multiple types of character codes for one category,
The sample data selection unit
For each category, sample data used for learning is selected so that the number of types of character codes is maximized and the difference in the number of sample data selected between character codes is minimized within the specified number of selections. The data processing apparatus according to claim 7. The classification unit includes:
Classify the data entered for each period into one of multiple categories,
The sample data selection criteria information storage unit is
Stores sample data selection criteria information in which the total number of sample data selected is a value obtained by multiplying the number of data input in each cycle by a predetermined ratio,
The sample data selection unit
Based on the sample data selection criteria information, a value obtained by multiplying the number of input data by the predetermined ratio is calculated as the total number of selected sample data for each period, and the learning is performed using the calculated total number of selected sample data. The data processing apparatus according to claim 1, wherein sample data used for part learning is selected. The computer classifies the data into one of several categories according to the classification rules,
The computer performs learning using sample data, and newly generates a classification rule used for data classification,
The computer extracts sample data used for the learning for each category,
Based on the sample data selection criteria information that shows the upper limit of the total number of sample data selections and the sample data selection criteria, the computer determines that the total number of sample data selections reaches the maximum within the range of the upper limit values and between categories. A data processing method characterized by calculating a selection number that minimizes a difference for each category, and selecting sample data used for learning for each category according to the selection number for each category from the extracted sample data. A classification process for classifying data into one of multiple categories according to a classification rule;
A learning process that performs learning using sample data and newly generates a classification rule used for the classification process;
Sample data extraction processing for extracting sample data used for learning of the learning processing for each category;
A read process for reading sample data selection criteria information indicating the upper limit of the total number of sample data selections and the sample data selection criteria;
Based on the sample data selection criteria information, the number of selected sample data is maximized within the range of the upper limit value, and the number of selections that minimizes the difference between categories is calculated for each category, and extracted by the sample data extraction process. A program for causing a computer to execute sample data selection processing for selecting sample data to be used for learning in the learning processing for each category according to the number of selected categories in the sample data.

Images