JP4994199B2 - Machine learning apparatus and machine learning method - Google Patents

Description

  The present invention relates to a machine learning device and a machine learning method.

In support vector machine, which is one of the identification methods using supervised learning in the field of machine learning, artificially create a virtual case near the support vector and add it to the case set used for learning Has been found to contribute to accuracy improvement (for example, Patent Document 1 and Non-Patent Document 1).
JP 2004-287776 A “Natural Language Processing”, Language Processing Society of Japan, 2006 Vol. 13, no. 3, pages 21-35

  However, the method of creating an effective virtual case is not always clear, and artificially creating a virtual case is laborious and the number of cases that can be created is limited. On the other hand, if the target of the support vector machine is limited to the problem of document classification, the document as an example can be easily obtained from the Web, and these can be used as a substitute for the above-described artificial operation. If possible, the accuracy can be improved without much effort.

  Therefore, an object of the present invention is to provide a machine learning device and a machine learning method for improving accuracy of a support vector machine related to document classification using a document acquired from the Web.

  The present inventor introduced the concept of distance (closeness) between a labeled case (document) that is a teacher of a support vector machine related to document classification and a case (document) acquired from the Web, and the distance Has found a mechanism to label the case (document) when a certain condition is satisfied and add it to the teacher of the support vector machine, and has completed the present invention. Specifically, the present invention provides the following.

(1) When determining whether or not an unknown case has the specific attribute as to whether or not the specific attribute is provided, based on known information about the data related to the case and the attributes of the case In the machine learning device to improve the determination accuracy,
Case acquisition means for acquiring a case that is unknown as to whether or not the specific attribute is provided;
Distance evaluation means for evaluating the distance between the case acquired by the case acquisition means and the case known to have the specific attribute;
For the case where the distance evaluated by the distance evaluation means satisfies a predetermined condition, the attribute giving means to consider that the specific attribute is provided and to add to the determination criterion;
A machine learning device comprising:

  According to such a configuration of the present invention, whether the machine learning device has the specific attribute on the basis of known information about data relating to the case and whether or not the case has the specific attribute. When determining whether or not an unknown case has the specific attribute, an unknown case is acquired as to whether or not the specific attribute is provided, and whether or not the case and the specific attribute are provided is known When the distance from the case is evaluated and the distance satisfies a predetermined condition, the case is regarded as having a specific attribute and used as a criterion for determination.

  As a result, the number of cases that can be used as a reference without relying on humans increases, so that the determination accuracy can be automatically improved.

  (2) The case is characterized as the data set, and the information about whether the case has a specific attribute is a label indicating whether the case belongs to a specific category. The machine learning device according to claim 1.

  According to such a configuration of the present invention, the machine learning device is useful for improving the determination accuracy for a support vector machine to which training data (example) is given as a combination of a vector and a label that is a data set. Can do. Therefore, the known information about the data related to the case and whether or not the case has a specific attribute corresponds to the labeled case.

  (3) The machine learning apparatus according to claim 1 or 2, wherein the case is a document, and the case acquisition unit acquires the document from the Web.

  According to such a configuration of the present invention, the machine learning device can use a document acquired from the Web, particularly in the case of a document classification problem, so that the richness of document resources on the Web can be improved. Can be useful.

  (4) The machine learning apparatus according to claim 3, wherein the document acquired by the case acquisition unit is a document created after a predetermined time.

  According to such a configuration of the present invention, since the machine learning device excludes a document with an older creation time from a large number of documents on the Web, the terminology and expression of the document change with the times. It is possible to respond appropriately.

  (5) The machine learning device according to claim 3 or 4, wherein the case acquisition unit acquires the document from a Web page related to the specific attribute.

  According to such a configuration of the present invention, the machine learning device acquires a document from a website or the like related to the attribute to be determined, not a document arbitrarily acquired from the Web. The error of the attribute of the document provided by the means can be reduced, which can further contribute to the improvement of the accuracy of the support vector machine.

  (6) The data is a word included in the document, and the distance evaluation unit is configured to acquire the distance, information related to the appearance of the word included in the document known to have the specific attribute, and the case acquisition. 6. The machine learning apparatus according to claim 3, wherein the evaluation is performed based on information relating to appearance of a word included in the document acquired by the means.

  According to such a configuration of the present invention, the distance evaluation unit calculates the distance between a case as a teacher (a document known to have the specific attribute) and a case (document) acquired from the Web, respectively. Since the evaluation is based on the information related to the appearance of the word included in the document, it is possible to objectively evaluate in consideration of the redundancy of the word and the appearance frequency. As a result, it is possible to determine that documents having similar characteristics in the appearance of words belong to the same category.

  (7) The data is a word included in the document, and the distance evaluation means uses the distance as information related to the appearance of the word included in the document known to have the specific attribute, and the specific The evaluation is based on information related to the appearance of a word included in a document that is known not to have an attribute, and information related to the appearance of a word included in the document acquired by the case acquisition unit. The machine learning device according to claim 3.

  According to such a configuration of the present invention, the distance evaluation unit calculates a distance between a case as a teacher (a document known to have the specific attribute) and a case (document) acquired from the Web. Information related to the appearance of a word contained in a document known to have the specific attribute, information related to the appearance of a word contained in a document not known to have the specific attribute, and the case acquisition Since evaluation is performed based on information related to the appearance of a word included in a document acquired by means, not only a case as a teacher (a document that is known to have the specific attribute) but also a case as an anti-teacher ( Documents that are known not to have the specific attribute can also be considered and evaluated. As a result, the eligibility of a document acquired from the Web as a teacher can be more safely recognized.

(8) In a case where it is determined whether an unknown case has the specific attribute as to whether or not the specific attribute is provided with reference to known information about the data related to the case and the attribute of the case A machine learning method for improving determination accuracy,
A case acquisition step of acquiring a case unknown about whether or not the computer has the specific attribute;
A distance evaluation step for evaluating a distance between the acquired case and a case known to have the specific attribute;
An attribute evaluation step for adding to the determination criteria by regarding the case where the distance evaluated by the computer satisfies a predetermined condition as being provided with the specific attribute;
A machine learning method comprising:

  According to this invention, since the process performed by the invention described in (1) is performed using a computer, the same effect as that of the invention described in (1) can be exhibited.

  According to the present invention, the accuracy of determination of a support vector machine related to document classification is improved by using a large number of documents on the Web, instead of artificially creating virtual cases as in the prior art. Therefore, a highly accurate document classification system can be created without taking time and effort.

Hereinafter, the best mode for carrying out the present invention will be described with reference to the drawings. This is merely an example, and the technical scope of the present invention is not limited to this.
(First embodiment)

[Overall view]
FIG. 1 is a diagram illustrating a relationship between the functional configuration of the machine learning device 1 and other elements related to the device 1. The apparatus 1 includes a case acquisition unit 2, a distance evaluation unit 3, and an attribute assignment unit 4.

  The case acquisition unit 2 acquires a document via the Internet 7. In this case, there is no need to limit the document to be acquired, but the document creation time may be limited to those after a predetermined date. This is because old documents may not be appropriate as training examples that help improve the accuracy of support vector machines.

  The distance evaluation means 3 checks the appearance status of the words included in the document received as described above, and the appearance of the words included in the known document whether or not it belongs to a specific category recorded on the labeled document storage device 5 Evaluate whether both documents are close by comparing with the situation. The comparison target includes the overlapping degree and appearance frequency of the appearing words. The evaluation of the distance may be expressed quantitatively by a numerical value, or may be expressed qualitatively such as “near” or “far”.

  When the distance evaluated above satisfies a predetermined condition, the attribute assigning unit 4 regards the acquired document as belonging to the category and records it in the labeled document storage device 5. By doing so, the document classification device 6 can improve the determination accuracy as a reference, that is, a training example, by adding the document to a document that is known to belong to a specific category.

  The labeled document storage device 5 is accessible from the machine learning device 1 and records a document that is already known whether or not it belongs to a specific category, that is, an existing training example. The labeled document storage device 5 may be in the machine learning device 1, but is outside the device 1 in the present embodiment.

  The document classification device 6 is connected to the labeled document storage device 5. The document classification device 6 uses a document that is known whether it belongs to a specific category recorded in the device 5 as a reference, that is, a training example. Perform classification.

  The internet 7 is accessible from the machine learning device 1, and the device 1 accesses the web page 8 via the internet 7.

[Hardware configuration of machine learning device 1]
FIG. 2 is a diagram illustrating a hardware configuration of the machine learning device 1 according to the present embodiment. The machine learning device 1 includes a central processing unit (CPU) 21 (a plurality of CPUs such as a CPU 22 may be added in a multiprocessor configuration), a bus line 10, and a communication I / F (I / F: An interface) 23, a main memory 24, a BIOS (Basic Input Output System) 25, a display device 26, an I / O controller 27, and an input device 28 such as a keyboard and a mouse.

  The control device 20 is a device for comprehensively controlling the machine learning device 1, and cooperates with the above-described hardware by appropriately reading and executing various programs stored in the hard disk 29 (described later), Various functions according to the present invention are realized.

  The communication I / F 23 is a network adapter that is used when the machine learning device 1 acquires a document of the Web page 8 via the Internet 7. The communication I / F 23 may include a modem, a cable modem, and an Ethernet (registered trademark) adapter. The BIOS 25 records a boot program executed by the CPU 21 when the machine learning device 1 is started up, a program depending on the hardware of the machine learning device 1, and the like.

  The display device 26 displays a screen such as a calculation processing result by the machine learning device 1 and includes a display device such as a cathode ray tube display device (CRT) or a liquid crystal display device (LCD). A storage device 31 such as a hard disk 29 and a semiconductor memory 30 can be connected to the I / O controller 27. The input device 28 accepts input by the administrator of the machine learning device 1. The hard disk 29 stores various programs for causing the hardware to function as the machine learning device 1, a program for executing the functions of the present invention, and a table to be described later.

  In the above example, the hardware configuration of the machine learning device 1 has been mainly described. However, the functions described above are realized by installing a program in a computer and operating the computer as the machine learning device 1. You can also. Therefore, the functions realized by the machine learning device 1 described as an embodiment in the present invention are performed by executing the above-described method by the computer, or by introducing the above-described program into the computer and executing it. Is also feasible.

  The computer referred to in the present invention refers to an information processing device including a storage device, a control device, and the like, and the machine learning device 1 includes an information processing device including a storage device 31, a control device 20, and the like. The information processing apparatus is included in the concept of the computer of the present invention. Here, the control device 20 mainly corresponds to the case acquisition means 2, the distance evaluation means 3, and the attribute assignment means 4 shown in FIG.

[Document vectorization]
FIG. 3 is a word dictionary used for grasping the appearance status of words in a document. It consists of a word number and a word. In the present embodiment, the word is limited to a noun.

  FIG. 4 is an appearance frequency table of words in a document related to “car”. When a document is given, words included in the document can be aggregated for each appearance frequency using a word dictionary (FIG. 3). In the case where there are a plurality of documents that are known to be related to “car”, the frequency of appearance of words included in these documents is totaled. In addition, the ratio of appearance is also calculated. The appearance ratio may be, for example, the number of times each word appears in the total number of words, or the number of times each word appears in the total number of characters. Any other calculation method may be used as long as it indicates the relative ease of appearance of each word.

  In this way, if the appearance frequency of the word in the document relating to “car” is known, a word having an appearance ratio exceeding a predetermined ratio is selected. For example, if a word having an appearance ratio exceeding 7% is selected, “engine”, “handle”, and “meter” are selected. The respective appearance ratios are 13%, 11%, and 9%. Then, the appearance ratio of these three words in each document is obtained using the three words thus selected as coordinate axes. Then, each document can be represented as a three-dimensional space point (ie, a three-dimensional vector).

Even documents about "cars", all does not mean the same ratio, for example, a document is "engine" 11% "handle" 10%, so called "meter" 11%. Another document is, for example, “Engine” 14%, “Handle” 11%, “Meter” 8%. However, on average, it is expected to be distributed around points represented by “engine” 13%, “handle” 11%, and “meter” 9%.

  On the other hand, if a document that is known to be related to “car” is labeled as “+1”, and a document that is known not to be related to “car” is labeled as “−1”, Each document for which it is known whether or not it is a document relating to a “car” can be grasped as a set of a vector and a label. Then, based on the theory of support vector machines, it is possible to obtain an optimal hyperplane that separates documents (cases) with different labels using these as training examples. Along with this, it is possible to grasp a document (example) involved in determining the optimum hyperplane as a support vector. The method for determining the optimum hyperplane and the method for determining the support vector are well known and will be omitted (see Non-Patent Document 1). In the above example, the document is vectorized based on the appearance frequency of the word. However, it is of course possible to vectorize the document depending on whether or not a predetermined word appears in the document.

[Distance between document obtained from Web and support vector]
Similarly, a document acquired from the Web (hereinafter referred to as an acquired document) can be grasped as a vector. In this case, since the object of the problem is narrowed down to whether it is a document related to “car”, it is grasped as a vector in a three-dimensional space having the appearance ratio of “engine”, “handle”, and “meter” as coordinate axes.

  The distance between two vectors in the three-dimensional space can be defined as the square root (Euclidean distance) of the sum of the squares of the differences between the elements. However, the present invention is not limited to this, and may be defined as, for example, the sum of absolute values of differences between elements. How to define the distance may be determined as appropriate according to the target problem.

  FIG. 5 is a diagram showing a labeled document (training example) and an optimal hyperplane. A white circle represents a document with a label “+1”, and a black circle represents a document with a label “−1”. A hyperplane that separates documents with different labels is a straight line 40, and a white circle 41 and a black circle 42 represent documents that are support vectors.

  FIG. 6 is a diagram in which a document acquired from the Web is entered. Here, the acquired document is indicated by a square 43. The distance d between the acquired document and the support vector is measured, and if this satisfies a predetermined condition, it is determined that the distance between the acquired document and the support vector is sufficiently close, and the same label as the support vector is assigned to the document. In this case, since the distance from the support vector with the label “+1” is short, the label “+1” of the document is set.

[Reflection to training examples]
A support vector machine learns a combination of a case represented by a vector and a label that is an index indicating whether or not the case belongs to a specific category, based on a plurality of training cases. I do.

  Therefore, a document with a label can be a training example. Therefore, an acquired document that is evaluated as being close to the support vector and assigned the same label as the support vector can be used as a training example. As a result, the optimal hyperplane from which the document set incorporating this acquired document as a training example is derived as a new training example can be different from the previous optimal hyperplane.

  FIG. 7 is a diagram showing changes in the optimum hyperplane. The conventional optimum hyperplane is shown as a broken line 40, and the new optimum hyperplane is shown as a solid line 50. Under the new optimal hyperplane, the support vector may change, but if the same operation is repeated on a new document obtained from the Web based on the new support vector, the number of training cases will increase. Therefore, it can be expected that the accuracy of attribute determination of documents related to “cars” will gradually improve.

[Processing flow]
The flow of the machine learning process will be described with reference to FIG. First, a plurality of documents labeled with whether or not the category is “car” is given, and learning for document classification is performed based on these documents in the normal procedure of the support vector machine (S10). ). At this time, based on these documents, a word for grasping the document as a vector and a document serving as a support vector are determined.

  Next, a document is acquired from the Web (S20). That is, the machine learning device 1 functions as a case acquisition unit.

  Next, the distance between the acquired document and the document serving as a support vector is evaluated (S30). That is, the machine learning device 1 functions as a distance evaluation unit.

  Next, it is determined whether the distance satisfies a predetermined condition, and when it is determined that the distance is satisfied (S40: Yes), the same label as the support vector is attached to the acquired document (S50). Then, the acquired document is added to a set of labeled documents, and learning for document classification is performed again by the normal procedure of the support vector machine based on these documents (S60). That is, the machine learning device 1 functions as an attribute assigning unit.

  If the above distance does not satisfy the predetermined condition (S40: No), the acquired document is terminated without performing anything.

  As described above, according to the present embodiment, a document is vectorized, and whether or not the document belongs to a category is labeled, so that it becomes a target of processing of the support vector machine. Then, using the feature that the label can only move discretely while the vector can move continuously on the real value, a virtual case is artificially created near the support vector. It becomes possible to apply the known knowledge that accuracy is improved by adding to a training example to a document acquired from the Web.

  This makes it possible to improve the accuracy of a support vector machine related to document classification using abundant documents existing on the Web without taking time and effort.

(Second Embodiment)
In the first embodiment, when vectorizing a document, an appearance is generated based on the appearance frequency of words in a plurality of documents that are known to be documents related to “car” (that is, a document with a label “+1”). A word with a high ratio was selected and used as a coordinate axis. However, in the present embodiment, the coordinate axis is selected in consideration of the appearance frequency of words in a plurality of documents that are known not to be related to “car” (that is, a document whose label is “−1”).

  Note that, in the following description and drawings, the same reference numerals are given to portions that perform the same functions as those of the first embodiment described above, and redundant descriptions are omitted as appropriate.

  FIG. 9 is a word frequency table in a document whose label is “−1”.

  FIG. 10 compares the appearance ratios in FIG. 4 and FIG. It can be seen from this table that “engine”, “handle”, and “meter” often appear in documents whose label is “+1”, but “mouse” and “line” rarely appear. Therefore, by setting the coordinate axes of vectorization of the document to “engine”, “handle”, “meter”, “mouse”, and “line”, not only words that should appear but also words that should not appear. Judgment in consideration of the appearance situation is possible.

  In this case, each document is expressed as a vector in a five-dimensional space. In this way, when a word “mouse” or “line” that does not appear in the document with the label “+1” appears in the document acquired from the Web, the label “+1” is supported in the five-dimensional space. The distance from the vector is increased, and as a result, the possibility of being included in a training case is reduced. That is, it is possible to expect a judgment with higher accuracy than the method shown in the first embodiment.

  As described above, according to the present embodiment, in vectorization of a document, not only a document with a positive label but also a document with a negative label is used as a reference to determine an element to be a vector. Vectorization becomes possible.

  As a result, a document that is truly suitable as a training example can be acquired from the Web.

  As mentioned above, although embodiment of this invention was described, this invention is not restricted to embodiment mentioned above. For example, the following method can be used to determine the distance between a document acquired from the Web and a document with a known label.

FIG. 11 is a diagram illustrating the distance between a document newly acquired from the Web and a document whose label is “+1”. Label indicates the document "+1" by a white circle in FIG. 1 1 (a). Each document is vectorized, and the extension of the set of vectors is indicated by a straight line. Therefore, the polygon surrounded by this straight line is the smallest polygon including these vectors.

Here, indicating acquired document by a black circle, the distance between the set of the black circles and white circles, as shown in FIG. 1 1 (b), a white circle constituting the vertices of extension of the set of open circles (hereinafter referred to as nodes) And can be defined as the distance.

The present invention is not limited to the distance between nodes, as shown in FIG. 1 1 (c), the sides of the extension from the closest point (black circle in the sides of the black circles and extension indicating the newly obtained documents from the Web If the distance between the vertically suspended line and the point where the side intersects is smaller than the distance between the black circle and any node, this may be the distance between the black circle and the set of white circles. This is because it is considered that a document having a short distance from the boundary of the closed region is considered appropriate even if the label is regarded as “+1”.

  In this way, the distance between the document acquired from the Web and the document with the label “+1” is calculated as appropriate based on the distance to the node or side, and it is determined whether this distance satisfies a predetermined condition. .

  According to such a method, since the distance between a document acquired from the Web and a document with a known label can be flexibly considered within a reasonable range, a document to be originally added to the training case is leaked. Can get without.

  As described above, in order to determine whether it is appropriate to attach a label to a document acquired from the Web, various methods for evaluating the distance from a document with a known label can be considered. Which method should be used may vary depending on the type of target document and the problem to be determined. Various methods for calculating the distance are conceivable, and various criteria for evaluating the distance are conceivable. All of these should be optimized through actual examples in the future, and these are all included in the technical scope of the present invention.

  Note that documents may be collected from a website related to a category to be determined by the support vector machine. For example, documents collected from places such as blogs that post opinions about "cars" are almost certainly considered to be documents about "cars", so various forms of documents related to "cars" are used as training examples. It is because it becomes possible to collect as.

  It should be noted that the effects described in the embodiments of the present invention only list the most preferable effects resulting from the present invention, and the effects of the present invention are limited to those described in the embodiments of the present invention. It is not a thing.

It is a figure which shows the relationship with the other element relevant to the function structure of the machine learning apparatus 1 which concerns on the 1st Embodiment of this invention. It is a figure which shows the hardware constitutions of the machine learning apparatus 1 which concerns on the 1st Embodiment of this invention. It is a figure which shows the word dictionary which concerns on the 1st Embodiment of this invention. It is a figure which shows the word frequency table which concerns on the 1st Embodiment of this invention. It is a figure which shows the labeled document (training example) and optimal hyperplane which concern on the 1st Embodiment of this invention. FIG. 5 is a diagram in which a document acquired from the Web is entered. It is a figure which shows the change of the optimal hyperplane which concerns on the 1st Embodiment of this invention. It is a figure which shows the flow of the process of the machine learning which concerns on the 1st Embodiment of this invention. It is a figure which shows the word frequency table which concerns on the 2nd Embodiment of this invention. FIG. 10 is a comparison of appearance ratios in FIG. 4 and FIG. 9. It is a figure which shows an example of the distance of the document newly acquired on Web and the document whose label is "+1".

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Machine learning apparatus 2 Case acquisition means 3 Distance evaluation means 4 Attribute assignment means 5 Labeled document storage device
6 Document classification device 7 Internet 8 Web page 10 Bus line 20 Control device 21, 22 CPU (Central Processing Unit)
23 Communication I / F (I / F: Interface)
24 main memory 25 BIOS (Basic Input Output System)
26 Display Device 27 I / O Controller 28 Input Device 29 Hard Disk 30 Semiconductor Memory 31 Storage Device

Claims (8)

When determining whether or not an unknown case has the specific attribute, whether or not the specific case has the specific attribute on the basis of the known information about the data related to the case and the attribute of the case A machine learning device for improving
Case acquisition means for acquiring a case that is unknown as to whether or not the specific attribute is provided;
Distance evaluation means for evaluating the distance between the case acquired by the case acquisition means and the case known to have the specific attribute;
For the case where the distance evaluated by the distance evaluation means satisfies a predetermined condition, the attribute giving means to consider that the specific attribute is provided and to add to the determination criterion;
Equipped with a,
The distance evaluation means has a first distance, which is a distance between the case acquired by the case acquisition means and a closest point of an edge constituting an extension of the case known to have the specific attribute, When the distance is smaller than a second distance that is a distance to any case that is known to have a specific attribute, the first distance is set as a distance evaluated by the distance evaluation means.
Machine learning device characterized by   The case is characterized by the data set, and the information about whether the case has a specific attribute is a label indicating whether the case belongs to a specific category. The machine learning device according to 1. The case is a document,
The machine learning apparatus according to claim 1, wherein the case acquisition unit acquires the document from the Web.   The machine learning apparatus according to claim 3, wherein the document acquired by the case acquisition unit is a document created after a predetermined time.   The machine learning apparatus according to claim 3, wherein the case acquisition unit acquires the document from a Web page related to the specific attribute. The data is a word contained in the document;
The distance evaluation means is information related to the appearance of a word included in a document known to have the specific attribute, and information related to the appearance of a word included in the document acquired by the case acquisition means. The machine learning apparatus according to claim 3, wherein the machine learning apparatus evaluates based on: The data is a word contained in the document;
The distance evaluation means includes the distance, information relating to the appearance of a word contained in a document known to have the specific attribute, and a word contained in a document known not to have the specific attribute. The machine learning according to any one of claims 3 to 5, wherein the evaluation is performed on the basis of information relating to the appearance of a word and information relating to the appearance of a word included in the document acquired by the case acquisition means. apparatus. In the case where it is determined whether or not an unknown case has the specific attribute with respect to whether or not the specific attribute is provided with reference to known information about the data related to the case and the attribute included in the case, the determination accuracy is A machine learning method to improve,
A case acquisition step of acquiring a case unknown about whether or not the computer has the specific attribute;
A distance evaluation step for evaluating a distance between the acquired case and a case known to have the specific attribute;
An attribute evaluation step for adding to the determination criteria by regarding the case where the distance evaluated by the computer satisfies a predetermined condition as being provided with the specific attribute;
Equipped with a,
In the distance evaluation step, a first distance that is a distance between a case acquired by the case acquisition step and a closest point of a side constituting an extension of the case that is known to have the specific attribute is If the distance is smaller than a second distance that is a distance to any case that is known to have a specific attribute, the first distance is the distance evaluated by the distance evaluation step.
A machine learning method characterized by

Images