JP2013131073A - Classification model learning method, device, program, and review document classifying method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To properly classify whether documents are review documents or not, from the small amount of learning data.SOLUTION: A sentence dividing portion 22 divides each of a plurality of blog documents for learning which include review documents, in which information about a specified object is written, in a sentence unit. A morphological analyzing portion 23 performs morphological analysis processing to each sentence, and an identity extracting portion 24 extracts identities indicating characteristics of the sentence about each sentence of the blog documents for learning. A learning portion 25 uses each of the extracted identities about each sentence of the blog documents for learning which are the review documents as learning data of positive examples, and each of the extracted identities about each sentence of the blog documents for learning which are not the review documents as learning data of negative examples, thereby learning a classification model for classifying whether the inputted sentences are sentences in the review documents or not.

Description

  The present invention relates to a classification model learning method, apparatus, program, and review document classification method, and more particularly to a classification model learning method, apparatus, program, and review document classification for classifying whether or not a document is a review document. Regarding the method.

  There is a classification method for classifying a document using a word distribution in a document such as a blog (for example, Non-Patent Document 1). In this classification method, there are words that are likely to appear in a specific field and words that are not so, and a document is classified for each field by using their distribution.

Koichi Hirano, Kiya Kobayashi, Junichi Takahashi, "Automatic classification of Japanese-speaking blogs", IPSJ Research Report, 2005

  However, when classifying whether or not a document obtained on social media such as a blog is a review document, it is very important to create a large amount of learning data when creating a classifier because the document is written in various formats In addition, there is a problem that it is difficult to appropriately classify whether or not the document is a review document with a small amount of learning data.

  The present invention has been made in view of the above circumstances, and a classification model learning method, apparatus, program, and review document classification method capable of appropriately classifying whether or not the document is a review document with a small amount of learning data The purpose is to provide.

  In order to achieve the above object, the classification model learning method according to the present invention includes a step of dividing each of a plurality of learning documents including a review document in which information relating to a specific object is described by a sentence dividing unit. Extracting a feature indicating the feature of the sentence for each sentence of the learning document divided by the sentence dividing means by the feature extracting means; and each sentence of the plurality of learning documents by the learning means. Learning a classification model for classifying whether the input sentence is a sentence in the review document based on each of the features extracted for.

  The classification model learning device according to the present invention includes a sentence dividing unit that divides each of a plurality of learning documents including a review document in which information about a specific target is described, and the sentence dividing unit that is divided by the sentence dividing unit. Based on each of the features extracted for each sentence of the plurality of learning documents, the feature extraction means for extracting a feature indicating the feature of the sentence for each sentence of the learning document Learning means for learning a classification model for classifying whether or not the sentence is in a review document.

  According to the classification model learning method and the classification model learning apparatus according to the present invention, the sentence dividing unit divides each of the plurality of learning documents including the review document in which the information about the specific target is described in sentence units. A feature indicating the feature of the sentence is extracted for each sentence of the learning document divided by the sentence dividing means by the feature extracting means.

  And a classification model for classifying whether or not the input sentence is a sentence in the review document based on each of the features extracted for each sentence of the plurality of learning documents by the learning unit. To learn.

  In this way, by dividing each of the learning documents including the review document into sentence units and learning the classification model based on each of the features extracted for each sentence of the learning document, a small amount of learning data can be obtained. Therefore, it is possible to obtain a classification model that can appropriately classify whether the document is a review document.

  The review document classification method according to the present invention includes a step of dividing an input document by a sentence unit by an input sentence dividing unit, and each sentence of the document divided by the input sentence dividing unit by an input feature extracting unit. On the basis of the step of extracting the features, the classification model learned by the classification model learning method by the classification means, and the features of each sentence extracted by the input feature extraction means For each sentence of the above, the step of classifying whether or not it is a sentence in the review document, and the input document based on the classification result of each sentence of the document classified by the classification means by the determination means Determining whether or not is a review document.

  According to the review document classification method of the present invention, an input document is divided into sentence units by an input sentence dividing unit. The feature is extracted for each sentence of the document divided by the input sentence dividing means by the input feature extracting means.

  Whether each sentence of the document is a sentence in the review document based on the classification model learned by the classification means and the feature of each sentence extracted by the input feature extraction means. Classify. A determination unit determines whether the input document is the review document based on a classification result of each sentence of the document classified by the classification unit.

  In this way, by using the classification model learned based on each feature extracted for each sentence of the learning document including the review document, by determining whether the input document is a review document, It is possible to appropriately classify whether or not the document is a review document with a small amount of learning data.

  The program according to the present invention is a program for causing a computer to execute each step of the classification model learning method or the review document classification method.

  As described above, according to the classification model learning method, apparatus, and program of the present invention, each of the learning documents including the review document is divided into sentence units, and the features extracted for each sentence of the learning document are extracted. By learning the classification model based on each of them, it is possible to obtain a classification model that can appropriately classify whether or not the document is a review document with a small amount of learning data.

  Further, according to the review document classification method and program of the present invention, an input document is a review document using a classification model learned based on each feature extracted for each sentence of a learning document including the review document. By determining whether or not it is, it is possible to appropriately classify whether or not the document is a review document with a small amount of learning data.

It is the schematic which shows the structure of the review document classification | category apparatus which concerns on embodiment of this invention. It is a figure which shows the blog document input. It is a figure for demonstrating a review sentence and a non-review sentence. (A) The figure which shows the input blog document, (B) The figure which shows the result divided | segmented into the sentence, (C) The figure which shows the feature extracted about each sentence. It is a figure which shows the classification result with respect to each sentence. It is a flowchart which shows the content of the learning process routine in the review document classification device which concerns on embodiment of this invention. It is a flowchart which shows the content of the document classification | category process routine in the review document classification | category apparatus which concerns on embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<System configuration>
The review document classification apparatus 100 according to the embodiment of the present invention receives a blog document obtained on social media (for example, a blog) and inputs objective information or subjective information on a specific target (for example, a store or a product). A determination result as to whether or not the document is a review document in which information (for example, opinions such as word-of-mouth information) is described is output. One blog document is text data composed of one or more sentences. The review document classification apparatus 100 is configured by a computer including a CPU, a RAM, and a ROM that stores a program for executing a learning processing routine and a document classification processing routine described later. It is configured as follows. As shown in FIG. 1, the review document classification apparatus 100 includes an input unit 10, a calculation unit 20, and an output unit 30.

  The input unit 10 receives a document group including a plurality of blog documents input as learning documents. For example, data as shown in FIG. 2 can be input as a blog document. The input unit 10 receives, for each blog document, teacher information indicating whether the blog document is a review document related to a specific target, together with the blog document as a learning document.

  Further, the input unit 10 accepts a blog document input as a classification target.

  Note that the input blog document may have been subjected to morphological analysis. In this case, morphological analysis units 23 and 32 described later can be omitted.

  The calculation unit 20 includes a document database 21, a sentence division unit 22, a morpheme analysis unit 23, a feature extraction unit 24, a learning unit 25, and a model storage unit 26.

  The document database 21 stores a document group including a plurality of blog documents as learning documents received by the input unit 10 and teacher information for each blog document.

  The sentence division unit 22 divides each blog document of the document group stored in the document database 21 into sentence units. A known technique may be used as the sentence division method. For example, the sentence may be divided at a place where punctuation marks or line breaks appear. In addition, when using a document obtained on social media such as a blog, emoticons, pictograms, and the like are often used for sentence separation. Therefore, pictograms and emoticons may be used for sentence breaks.

  For each blog document, the morpheme analysis unit 23 divides the sentence into words by morpheme analysis, which is an existing technology, and further adds a part of speech to each word for output. For example, the blog document says, “When I went to Omotesando, it was a great line, so when I took a peek, the release of a new product had begun. If you have "...", the result of the morphological analysis is "... (omitted) ... A company (noun) / no (case particle) / new product (noun) / Case particle) / buy (verb stem) / tsu (verb inflection ending) / te (verb suffix) / shima (verb stem) / i (verb inflection ending) / was (verb suffix) ... (omitted) ... "is obtained.

  For each blog document, the feature extraction unit 24 uses the result obtained by the morphological analysis for each divided sentence to create a feature indicating the feature of the sentence used for machine learning. For example, a frequency distribution (histogram) of morphemes in a sentence is used as a sentence feature.

  There may be many specific emotional expressions and evaluation expressions appearing in review documents. Therefore, the presence / absence and type of sensitivity expression / evaluation expression may be used as a feature. In addition, emoticons and pictograms may be used as features.

  Also, social media texts such as blogs may be evaluated across multiple sentences. For this reason, evaluation expressions of preceding and following sentences, morpheme frequencies, and the like may be used as features.

  As shown in FIG. 3, the feature extraction unit 24 stores all sentences in the blog document that is the review document as review sentences, and stores the features of the review sentences in a memory (not shown) as learning data of positive examples. Further, the feature extraction unit 24 sets all sentences in the blog document, which is a non-reviewed document, as non-reviewed sentences, and stores the features of the non-reviewed sentences in the memory as negative example learning data. In the case of a blog, there are about 10 sentences per document. For example, if learning is necessary for about 1000 sentences, 100 blog documents can be appropriately learned.

  The learning unit 25 is input by machine learning using positive learning data (features of review sentences) and negative learning data (features of non-review sentences) obtained from a document group that is a learning document. A classification model for classifying whether or not the sentence is a sentence in the review document is created and stored in the model storage unit 26. As the machine learning algorithm, for example, an algorithm such as support vector machine (SVM) or Markov Logic Network (MLN) can be used.

  The classification model stored in the model storage unit 26 stores, for example, numerical values of weights regarding each feature.

  In addition, the calculation unit 20 includes a sentence division unit 31, a morpheme analysis unit 32, a feature extraction unit 33, a classification unit 34, and a review document determination unit 35. The sentence dividing unit 31 is an example of an input sentence dividing unit, and the feature extracting unit 33 is an example of an input feature extracting unit.

  Similarly to the sentence dividing unit 22, the sentence dividing unit 31 divides the input blog document to be classified as shown in FIG. 4A into sentence units as shown in FIG. 4B.

  Similar to the morpheme analysis unit 23, the morpheme analysis unit 32 divides the sentence into words by morpheme analysis for each divided sentence of the blog document to be classified, and further outputs a part of speech to each word.

  As shown in FIG. 4C, the feature extraction unit 33 uses the result obtained by morphological analysis for each divided sentence, as in the feature extraction unit 24, as shown in FIG. Create features that characterize the sentence.

  For example, for each sentence of the blog document to be classified, the classification unit 34 uses, for example, a feature vector whose elements are numerical values obtained by multiplying a numerical value indicating the extracted feature by a corresponding weight of the classification model. Then, according to the algorithm of the support vector machine, it is classified whether or not the sentence is a sentence in the review document. Accordingly, as shown in FIG. 5, each sentence is classified as a review sentence or a non-review sentence.

  The review document determination unit 35 determines that the blog document to be classified is a review document when the ratio of the sentence classified as the review sentence is equal to or greater than the threshold value, and determines that the review document is not It is determined that the document is a review document.

  The determination result by the review document determination unit 35 is output from the output unit 30.

<Operation of review document classification device>
Next, the operation of the review document classification apparatus 100 according to the present embodiment will be described. First, when a document group which is a plurality of blog documents as learning documents and teacher information indicating whether each of the plurality of blog documents is a review document are input to the review document classification apparatus 100, a review is performed. The document grouping apparatus 100 stores the input document group and teacher information in the document database 21. Then, the review document classification apparatus 100 executes a learning process routine shown in FIG.

  First, in step S101, one blog document is extracted from the document database 21. In step S102, the sentence dividing unit 22 divides the blog document extracted in step S101 into sentences. In step S103, the morpheme analysis unit 23 performs a morpheme analysis process on each sentence of the blog document.

  In the next step S104, the feature extraction unit 24 extracts features for each sentence of the blog document based on the morpheme analysis result obtained in step S103, and in step S105, if the blog document is a review document. The feature of each sentence extracted in step S104 is stored in the memory as the feature of the review sentence (positive learning data), and if the blog document is a non-review document, each feature extracted in step S104 is stored. The feature of the sentence is stored in the memory as the feature of the non-reviewed sentence (negative example learning data).

  In step S106, it is determined whether or not the processing in steps S101 to S105 has been executed for all the blog documents stored in the document database 21, and blog documents that have not executed the processing in steps S101 to S105 are determined. If it exists, the process returns to step S101 to take out the blog document. On the other hand, if the processes of steps S101 to S105 have been executed for all blog documents, the process proceeds to step S107.

  In step S107, the learning unit 25 learns the classification model by machine learning using the positive example learning data and the negative example learning data stored in the memory. In step S108, the classification model is stored in the model storage unit 26. Is stored, and the learning processing routine is terminated.

  When the blog document to be classified is input to the review document classification device 100, the review document classification device 100 executes a document classification processing routine shown in FIG.

  First, in step S111, a blog document input by the input unit 10 is received. In step S112, the sentence dividing unit 31 divides the blog document input in step S111 into sentence units. In step S113, the morpheme analysis unit 32 performs a morpheme analysis process on each sentence of the blog document.

  In the next step S114, the feature extraction unit 33 extracts the feature for each sentence of the blog document. In step S115, for each sentence of the blog document by the classification unit 34, whether the sentence is a review sentence based on the features extracted in step S114 and the classification model stored in the model storage unit 26. Classify whether it is a non-reviewed sentence.

  In step S116, the review document determination unit 35 determines whether the blog document is a review document based on the ratio of the sentences classified as review sentences in step S115. In step S117, the determination result in step S116 is output by the output unit 30, and the document classification processing routine is terminated.

  As described above, according to the review document classification device according to the present embodiment, each of the learning blog documents including the review document is divided into sentence units, and each sentence of the blog document that is the review document is extracted. By learning each classification feature as a positive example learning data and learning each classification feature as a negative example learning data for each sentence of a non-reviewed blog document, a small amount of learning data can be used. Therefore, it is possible to obtain a classification model that can appropriately classify whether the document is a review document.

  In addition, by using the classification model learned based on each feature extracted for each sentence of the learning blog document including the review document, it is determined whether or not the input blog document is a review document. It is possible to appropriately classify whether or not the document is a review document with a small amount of learning data.

  Note that the present invention is not limited to the above-described embodiment, and various modifications and applications are possible without departing from the gist of the present invention.

  For example, each sentence of a learning blog document may be manually input as to whether it is a review sentence in which objective or subjective information about a specific object is described or a non-review sentence. . In this case, each feature extracted for each of the review sentences in each sentence of the learning blog document is set as a positive example of learning data, and each of the non-review sentences in each sentence of the learning blog document is set. The classification model may be learned by the learning unit using each of the extracted features as negative learning data.

  In the present specification, the embodiment has been described in which the program is installed in advance. However, the program can be provided by being stored in a computer-readable recording medium.

DESCRIPTION OF SYMBOLS 10 Input part 20 Arithmetic part 21 Document database 22, 31 Sentence division part 23, 32 Morphological analysis part 24, 33 Feature extraction part 25 Learning part 26 Model storage part 34 Classification part 35 Review document determination part 100 Review document classification apparatus

Claims (7)

Dividing each of a plurality of learning documents including a review document in which information on a specific object is described by a sentence dividing unit;
Extracting features indicating characteristics of the sentence for each sentence of the learning document divided by the sentence dividing means by a feature extracting means;
Learning a classification model for classifying whether or not the input sentence is a sentence in the review document based on each of the features extracted for each sentence of the plurality of learning documents by the learning means. And steps to
Classification model learning method including   The step of learning by the learning means sets each feature extracted for each sentence of the learning document that is the review document as a positive feature, and extracts each sentence of the learning document that is not the review document. The classification model learning method according to claim 1, wherein the classification model is learned using each of the features as a negative example.   In the learning step by the learning means, each of the features extracted for each of the review sentences in which information on the specific target is described in each sentence of the learning document is set as a positive example feature, The classification model learning method according to claim 1, wherein the classification model is learned by using each of the features extracted for each of the sentences that are not the review sentence among the sentences of the document as features of negative examples. Dividing the input document into sentence units by the input sentence dividing means;
Extracting the features for each sentence of the document divided by the input sentence dividing means by an input feature extracting means;
Based on the classification model learned by the classification model learning method according to any one of claims 1 to 3 by the classification unit and the features of each sentence extracted by the input feature extraction unit, Categorizing whether each sentence of the document is a sentence in the review document;
Determining whether or not the input document is the review document based on a classification result of each sentence of the document classified by the classification unit;
Review document classification method including   The classification model learning method according to claim 1, wherein the document is a blog document. Sentence dividing means for dividing each of a plurality of learning documents including a review document in which information on a specific object is described;
For each sentence of the learning document divided by the sentence dividing means, a feature extracting means for extracting a feature indicating the feature of the sentence;
Learning means for learning a classification model for classifying whether or not an inputted sentence is a sentence in the review document based on each of the features extracted for each sentence of the plurality of learning documents; ,
Model learning apparatus including   A program for causing a computer to execute the steps of the classification model learning method according to claim 1, 2, 3, or 5, or the review document classification method according to claim 4.