JP2006190229A - Opinion extraction learning device and opinion extraction classifying device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an opinion extraction learning device for precisely extracting the opinions of writers from documents to be investigated, and to provide an opinion extraction classifying device. <P>SOLUTION: The opinion extraction learning device comprises an attribute evaluation paired candidate extracting means 200 for inputting the documents to be investigated on which the opinions of the writers such as questionnaire results are described, grouping and extracting attribute evaluation paired candidates from the documents, and imparting the features of attribute expression and evaluation expression for every attribute evaluation paired candidate, a classifying-1 means 260 for inputting the grouped attribute evaluation paired candidates and a classification model 1 such as an identifying function Z1 and extracting positive example attribute evaluation paired candidates from the attribute evaluation paired candidates in the same group, and a a classifying-2 means 290 for inputting the positive example attribute evaluation paired candidates and a classifying model 2 such as an identifying function Z2 and processing the extraction of a positive example attribute evaluation pair from the positive example attribute evaluation paired candidates. Thus, the opinions of the writers are extracted from the documents to be investigated. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an opinion extraction learning device and an opinion extraction classification device for extracting a pair of an attribute expression and an evaluation expression as an opinion from an investigation target document.

  Various documents such as blogs, Web bulletin boards, E-mails, questionnaire results, inquiry data for company inquiry counters, and reports, etc., describe many opinions about many objects. Many of these opinions include opinions indicating positive or negative evaluations on specific subjects, as well as requests for reporting complaints and reports on improvements and problems.

  If it is possible to automatically extract the part where the author's opinion about a specific object is described from a document in which such various opinions are described, the reputation and criticism of the object, Customers' requests can be obtained automatically, pre-study before product purchase, sales of improved products reflecting the opinions, development of new products, etc., so marketing such as corporate market research It is important information.

  However, since these general documents are not originally intended to describe opinions themselves about the object, they contain a lot of information unrelated to opinions. Therefore, since information regarding opinions cannot be extracted automatically and accurately from such a document, there is a problem that it cannot be used for statistics as it is.

  As background technology related to opinion extraction, attributes (design, price, support, performance, etc.) related to a specific object included in the document and evaluations related to the attribute expression (good, wide, like, etc.) There is known a method of extracting a pair (attribute evaluation pair) of an expression indicating a possible value of an attribute using a predetermined rule (see, for example, Non-Patent Document 1).

  In the method described in Non-Patent Document 1, an extraction rule for determining whether a correct relationship exists between an attribute expression and an evaluation expression, and an opinion for determining subjective evaluation when there is a correct relationship between the attribute expression and the evaluation expression Two sex determination rules are used.

  The above-mentioned attribute expression is an attribute expression related to a specific target for the purpose of extracting opinions, such as “design”, “room”, “price”, “support”, “performance”, etc. This shows an expression representing a certain aspect.

  The evaluation expression is an expression indicating an evaluation of things, and indicates an evaluation expression regarding values that the attribute expression can take such as “good”, “like”, “wide”, “cheap”, and the like.

An attribute evaluation pair indicates a combination of an attribute expression and an evaluation expression that satisfy the following two requirements.
(Requirement 1) There is a correct relationship between the attribute expression and the evaluation expression. (Requirement 2) Indicates subjective evaluation. For example, when considering the attribute expression “design” evaluation expression “good”, the following (a) Satisfies but does not satisfy (b). (B) is because the object that “good” refers to is not “design”.
(A) Good design.
(B) Aside from design, it can be good.

On the other hand, when the attribute expression “design” and the evaluation expression “good” are considered similarly, the following (c) satisfies the requirement 2, but (d) does not indicate the requirement 2. This is because (d) is a question and (e) is a condition and does not indicate the subjective evaluation of the writer himself.
(C) The design is good.
(D) Is the design good?
(E) If the design is good, buy it.

  Opinion extraction indicates that a positive attribute evaluation pair (a combination of correct attribute expression and evaluation expression) is extracted from a document. The negative example attribute evaluation pair indicates a combination of an attribute expression and an evaluation expression that are not suitable for extraction as an opinion.

Also, an anaphoric analysis model is known that uses machine learning to extract antecedent and anaphoric pairs from context (see, for example, Non-Patent Document 2).
Kenji Tateishi, Shunichi Fukushima, Nozomi Kobayashi, Tetsuro Takahashi, Atsushi Fujita, Kentaro Inui, Yuji Matsumoto: Extraction of opinion information from a set of Web documents and creation of a summary based on the viewpoint, Information Processing Society of Japan Research Report, NL-163, p.1 -8 (2004) Soon, WM, Ng, HT and Lim, DCY: A Machine Learning Approach to Corefernce Resolution of Noun Phrases, Computational Linguistics, Vol.27, No.4, pp.521-544 (2001)

In the opinion extraction method of Non-Patent Document 1, a case where opinion extraction processing is performed by providing the following extraction rule (1) and opinion determination rule (2) will be considered. Note that “→” in (1) means that a dependency relationship exists. The extraction rule is a rule for extracting a combination that is related between [attribute] and [evaluation] as a correct answer.
Extraction rules (1) [Attribute] + "ga"-> [Evaluation]
Opinion judgment rule (2) [Evaluation] + "If"

When the attribute expression is “design” and the evaluation expression is “good”, the following sample sentences (a) to (e) are used by using the above extraction rule (1) and opinion judgment rule (2). Try the opinion extraction process.
(A) Good design.
(B) As for the design, it can be said that it is a good result.
(C) I want a good product with a different design.
(D) I want it if the design is good.
(E) There is an idea that the design is good, but I ...

For example, if you try to extract opinions from the above sample sentence,
The sentence (a) is extracted on the assumption that an attribute evaluation pair representing an opinion exists in order to match the extraction rule (1).
Since the sentences (b) and (c) do not match the extraction rule (1), an attribute evaluation pair representing an opinion is not extracted.
Although the sentence (d) matches the extraction rule (1), the attribute evaluation pair representing the opinion is not extracted because it matches the opinion determination rule (2).
Since the sentence (e) matches the extraction rule (1) and does not match the opinion determination rule (2), an attribute evaluation pair representing an opinion is extracted.

  Therefore, when the opinion extraction method of Non-Patent Document 1 is used, an attribute evaluation pair representing an opinion is extracted from the sentences (a) and (e).

  However, in the above sentences (a) to (e), sentences including attribute evaluation pairs to be extracted as opinions are (a) and (b). That is, in the sentences of (a) and (b), an opinion by a writer of [attribute] = “design” and [evaluation] = “good” of a specific target is described, but (c) The object that “design” points to is not “good”, and the sentences (d) and (e) describe the subjective evaluation and opinion of the writer that “design is good” for a specific object. Because it cannot be said.

  Therefore, even if the opinion extraction method of Non-Patent Document 1 is used, there is a problem in that an opinion cannot be extracted from the sample sentence (b) in which the opinion of the writer is described.

  In addition, when the opinion extraction method of Non-Patent Document 1 is used, an attribute evaluation pair that is not originally the opinion of the writer is erroneously extracted as an opinion from the sample sentence (e) where the opinion of the writer is not described. Was produced.

  As described above, in the opinion extraction method based on the predetermined rule described in Non-Patent Document 1, the relationship between [attribute] and [evaluation] is ruled and the rule based on subjective evaluation is comprehensively described manually. It is very difficult to do.

  In addition, in the anaphora analysis model described in Non-Patent Document 2, the anaphora analysis model itself is not intended to extract opinions, so that the writer's opinion cannot be extracted from the document as it is. Was produced. In addition, the extraction method based on machine learning is an incorrect solution (a combination in which there is no relationship between an antecedent and an anaphor) with respect to an example of a correct answer (a combination determined to be related between an antecedent and an anaphor). There is a tendency that there are many examples of combinations) determined to be, and the extraction performance is lowered. In general, in machine learning, when learning is performed using a case where the balance between the correct answer example and the incorrect answer example is extremely different, the extraction performance tends to decrease.

  The present invention has been made to solve the above-described problem. An attribute evaluation pair indicating that there is some relationship between [attribute] and [evaluation] is extracted from the document, and the extracted attribute evaluation pair is extracted. On the other hand, it is an object to provide an opinion extraction learning device and an opinion extraction classification device capable of extracting positive attribute evaluation pairs based on machine learning and extracting opinions from a document with high accuracy. .

  In order to achieve the above object, the learning device for opinion extraction of the present invention inputs a training document including a positive example attribute evaluation pair with a label that associates an attribute expression and an evaluation expression in the document, and includes the training document in the training document. Attribute evaluation pair candidates corresponding to combinations of existing attribute expressions and evaluation expressions are grouped and extracted for each training document, and for each attribute evaluation pair candidate, a phrase including attribute expressions and evaluation expressions is extracted. The part-of-speech of the phrase including the surface character string, attribute expression and evaluation expression, whether the phrase including the attribute expression and the phrase including the evaluation expression are in a dependency relationship, or the distance information between the attribute expression and the evaluation expression For each training document and training document that includes attribute evaluation pair candidate extraction means for performing processing to add feature information, and a positive example attribute evaluation pair with a label that associates the attribute expression with the evaluation expression in the document. The grouped attribute evaluation pair candidates are input, and the same attribute evaluation pair candidate as the positive example attribute evaluation pair is defined as the positive example attribute evaluation pair candidate. A training case creation 1 means for performing a process of defining an attribute evaluation pair candidate that is not an example attribute evaluation pair candidate as a negative example attribute evaluation pair candidate, a positive example attribute evaluation pair candidate, and a negative example attribute evaluation pair candidate are input, and an identification function Grouped for each training document and training document including a model creation 1 means for performing a process of creating a classification model 1 such as Z1 and a positive example attribute evaluation pair with a label that associates an attribute expression with an evaluation expression in the document. Input an attribute evaluation pair candidate and a classification model 1 such as a discriminant function Z1, and belong to a group composed of attribute evaluation pair candidates that do not include any of the same attribute evaluation pair candidates as the positive attribute evaluation pair. Attribute evaluation pair candidates are extracted as undetermined attribute evaluation pair candidates, and the categorized model 1 such as the discriminant function Z1 is used to classify the undetermined attribute evaluation pair candidates as positive example attribute evaluation pair candidates or negative example attribute evaluation pair candidates. A classification 1 means for processing, a training document including a positive example attribute evaluation pair with a label that associates an attribute expression and an evaluation expression in the document, and an attribute evaluation pair candidate grouped for each training document and a classification model 1 means The classified positive example attribute evaluation pair candidate is input, the positive example attribute evaluation pair candidate is replaced with the negative example attribute evaluation pair candidate, and the same attribute evaluation pair candidate as the positive example attribute evaluation pair is replaced with the positive example attribute evaluation pair. The training case creation 2 means for performing the process determined as a candidate, the positive example attribute evaluation pair candidate determined by the training case creation 2 means, and the negative example attribute evaluation pair candidate replaced by the training case creation 2 means are input. And a model creation 2 means for performing a process of creating and outputting a classification model 2 such as an identification function Z2.

  According to the present invention, a negative example is automatically extracted from a training document, and a negative model is selected so that the classification model 1 created by selectively using the negative example can be learned mainly in subjective evaluation. Since the classification model 2 created by using them is output, it is possible to learn the relationship between attribute expressions and evaluation expressions that are important for opinion extraction.

  In order to achieve the above object, the opinion extraction classification apparatus of the present invention is a web bulletin board such as blog, E-mail, questionnaire results, inquiry data for company inquiry window, report, other writers Input the survey target document in which the opinions are described, and extract the attribute evaluation pair candidates corresponding to the combination of the attribute expression and the evaluation expression existing in the survey target document by grouping for each target document , For each attribute evaluation pair candidate, the surface character string of the phrase including the attribute expression and the evaluation expression, the part of speech of the phrase including the attribute expression and the evaluation expression, the phrase including the attribute expression and the phrase including the evaluation expression Attribute evaluation pair candidate extraction means for performing processing for adding feature information such as distance information between the attribute expression and the evaluation expression, and each survey target document The grouped attribute evaluation pair candidate and the attribute evaluation pair candidate are input into a classification model 1 such as a discriminant function Z1 for determining a boundary surface for dividing the attribute evaluation pair candidate into a positive example attribute evaluation pair and a negative example attribute evaluation pair. Class 1 means for performing processing for extracting positive example attribute evaluation pair candidates from candidate attribute evaluation pair candidates in the same group, and positive example attribute evaluation pair candidates and positive example attribute evaluation pair candidates extracted by classification 1 means Input a classification model 2 such as a discriminant function Z2 for determining a boundary plane to be divided into an example attribute evaluation pair and a negative example attribute evaluation pair, and perform a process of extracting a positive example attribute evaluation pair from positive example attribute evaluation pair candidates It comprises classification 2 means for performing processing for determining the positive example attribute evaluation pair as a true attribute evaluation pair.

  According to the present invention, a classification model 1 created by selectively using negative examples automatically extracted from training documents and a negative model selectively created so that mainly subjective evaluation can be learned. By using the classification model 2, the opinion can be extracted from the survey target document using the result of effectively learning the relationship between the attribute expression and the evaluation expression.

  In order to achieve the above object, the opinion extraction learning apparatus of the present invention inputs a training document including a positive example attribute evaluation pair with a label that associates an attribute expression and an evaluation expression in the document, and enters the training document. Attribute evaluation pair candidates corresponding to combinations of existing attribute expressions and evaluation expressions are grouped and extracted for each training document, and the phrase surface layer including attribute expressions and evaluation expressions for each attribute evaluation pair candidate The part of speech of the phrase containing the character string, attribute expression and evaluation expression, whether the phrase including the attribute expression and the phrase including the evaluation expression are in a dependency relationship, or the features such as distance information between the attribute expression and the evaluation expression For each training document and training document that includes attribute evaluation pair candidate extraction means that performs processing for assigning information, and a positive example attribute evaluation pair with a label that associates the attribute expression with the evaluation expression in the document. The grouped attribute evaluation pair candidates are input to extract attribute evaluation pair candidates in a group including at least one attribute evaluation pair candidate identical to the positive attribute evaluation pair, and two sets of attribute evaluation pair candidates Attribute evaluation comparison pair candidates are generated based on the features and positional relationships of the two attribute expressions in the list, and the attribute expressions existing on the left side from the top in the training document are the attribute expressions of the positive example attribute evaluation pairs in the training document. If they match, an attribute evaluation comparison pair in which a Left label is assigned to the attribute evaluation comparison pair candidate is generated, and the attribute expression existing on the right side from the top in the training document is the correct example attribute evaluation pair in the training document. When the attribute expression matches, the training case creation 1 means for performing the process of generating the attribute evaluation comparison pair in which the right label is assigned to the attribute evaluation comparison pair candidate, and the training case creation 1 means Le A label for associating an attribute evaluation comparison pair with a label of t or Right with model creation 1 means for performing a process of creating a classification model 3 such as a discriminant function Z3, and an attribute expression in the document and an evaluation expression A training document including a positive example attribute evaluation pair with a symbol, an attribute evaluation pair candidate grouped for each training document, and a classification model 3 such as a discriminant function Z3 are input, and the same attribute evaluation pair as the positive example attribute evaluation pair Attribute evaluation pair candidates belonging to a group composed of attribute evaluation pair candidates that do not include any candidates are extracted as undetermined attribute evaluation pair candidates, and based on two attribute expression features in two undetermined attribute evaluation pair candidates The attribute evaluation comparison pair is generated, and the generated plurality of attribute evaluation comparison pairs are sequentially compared in the tournament method using the classification model 3 such as the discriminant function Z3. Class 1 means for performing a process of setting one remaining attribute evaluation pair candidate as a positive example attribute evaluation pair candidate and setting an attribute evaluation pair candidate other than the positive example attribute evaluation pair candidate as a negative example attribute evaluation pair candidate , A training document including a positive example attribute evaluation pair with a label that associates an attribute expression with an evaluation expression in the document, a candidate attribute evaluation pair grouped for each training document, and a positive example attribute evaluation pair classified by the classification model 1 means The candidate is input, the positive example attribute evaluation pair candidate is replaced with a negative example attribute evaluation pair candidate, and the process of determining the same attribute evaluation pair candidate as the positive example attribute evaluation pair as a positive example attribute evaluation pair candidate The classification model such as the discriminant function Z4 is inputted by inputting the case creation 2 means, the positive example attribute evaluation pair candidate determined by the training case creation 2 means and the negative example attribute evaluation pair candidate replaced by the training case creation 2 means. Characterized in that a model created second means for performing a process creates and outputs a.

  According to the present invention, a negative example is automatically extracted from a training document, and the negative model is selected so that the subjective evaluation can be learned mainly with the classification model 3 created by selectively using the negative example. Since the classification model 4 created by using the output is output, it is possible to learn the relationship between the attribute expression important for opinion extraction and the evaluation expression.

  In order to achieve the above object, the opinion extraction classification apparatus of the present invention is a web bulletin board such as blog, E-mail, questionnaire results, inquiry data for company inquiry window, report, other writers Input the survey target document in which the opinions are described, and extract the attribute evaluation pair candidates corresponding to the combination of the attribute expression and the evaluation expression existing in the survey target document by grouping for each target document , For each attribute evaluation pair candidate, the surface character string of the phrase including the attribute expression and the evaluation expression, the part of speech of the phrase including the attribute expression and the evaluation expression, the phrase including the attribute expression and the phrase including the evaluation expression Attribute evaluation pair candidate extraction means for performing processing for adding feature information such as distance information between the attribute expression and the evaluation expression, and each survey target document The grouped attribute evaluation pair candidate and the attribute evaluation pair candidate are classified into a positive example attribute evaluation pair and a negative example attribute evaluation pair, and a classification model 3 such as a discriminant function Z3 for determining a boundary surface is input. An attribute evaluation comparison pair is created from the two attribute evaluation pair candidates, and using the classification model 3 such as the discriminant function Z3, the generated plurality of attribute evaluation comparison pairs are compared in turn in the tournament method, and finally each group is finalized. Classification 1 means for performing processing for setting one attribute evaluation pair candidate remaining as a positive example attribute evaluation pair candidate and setting an attribute evaluation pair candidate other than the positive example attribute evaluation pair candidate as a negative example attribute evaluation pair candidate And a classification model such as a discriminant function Z4 for determining a boundary plane for dividing the positive example attribute evaluation pair candidate and the positive example attribute evaluation pair candidate set by the classification 1 means into a positive example attribute evaluation pair and a negative example attribute evaluation pair 4 is inputted, a process of extracting a positive example attribute evaluation pair from the positive example attribute evaluation pair candidates set by the classification 1 means, and a process of determining the positive example attribute evaluation pair as a true attribute evaluation pair And a classification 2 means.

  According to this invention, the classification model 3 created by selectively using negative examples automatically extracted from the training document and the negative model selectively created so that mainly subjective evaluation can be learned. By using the classification model 4, the opinion can be extracted from the survey target document using the result of effectively learning the relationship between the attribute expression and the evaluation expression.

  The effect of the present invention is to realize highly accurate opinion extraction. The reason is that the classification model 1 (or classification model 3) created by selectively using the negative example for the purpose of learning the relationship between the attribute and the evaluation, and the negative example for the purpose of learning the subjective evaluation property This is because the classification model 2 (or classification model 4) created selectively is used.

  The negative example attribute evaluation pair candidates input to the model creation 1 means for creating the classification model 1 (or classification model 3) are only negative examples of group IDs including one or more positive example attribute evaluation pairs of the training document. Since such a negative example can be determined to be a negative example in which there is no correct relationship between the attribute and the evaluation, the model creation 1 means can learn for the purpose of determining whether the attribute and the evaluation have a correct relationship. It is. On the other hand, the negative example attribute evaluation pair candidates input to the model 2 creation means for creating the classification model 2 (or the classification model 4) are only negative examples that are erroneously determined as the positive examples by the classification 1 means. Although it is determined to be a positive example by the classification 1 means, in the case of a negative example, it can be determined that it is a negative example that is not a subjective evaluation although there is a correct relationship between the attribute and the evaluation. Can be used for learning.

Thus, by selectively using negative examples in the classification model 1 and the classification model 2, learning with a clear objective can be performed, and the extraction accuracy can be improved. Even from Web bulletin boards, e-mails, report documents, and other documents that are not intended to describe opinions themselves, it is possible to automatically obtain reputation and criticism of the object, customer requests, etc. As a result, it is possible to extract the opinions of the writer more accurately and automatically. Therefore, it is possible to automatically obtain the reputation and criticism of the object, customer requests, etc., improve the efficiency of marketing activities such as pre-sale before product purchase and market research of companies, and more It becomes possible to provide products that comply with

  In the model of anaphora analysis described in Patent Document 2, the context of an antecedent and an anaphoric pair is replaced with the context of a pair of attribute expression and evaluation expression, and used as part of the process of extracting an attribute evaluation pair Seems to be able to. At present, various models of anaphora analysis using machine learning are known. However, the application of the simple model of anaphora analysis described in Patent Document 2 will be examined.

  Using this anaphora analysis model, an attribute evaluation pair is extracted, a binary classification problem of whether each element in the attribute candidate set is paired with the evaluation candidate, and a plurality of pairs of the evaluation candidate and the attribute candidate are solved. Is considered possible.

  In addition, among the attribute evaluation pairs (combinations determined to be related between [attribute] and [evaluation]) extracted as correct answers, there may be a plurality of pairs for one evaluation candidate. By adding a plurality of processes to this technique, there is a possibility that a pair of attribute expression and evaluation expression can be extracted comprehensively as an opinion of the writer for such a case.

  FIG. 1 is a block diagram relating to processing of a learning apparatus 10 that learns attribute evaluation pair candidate classifications from a training example including positive example attribute evaluation pairs input by an operator, and generates and outputs a classification model such as a discrimination function. is there.

  FIG. 2 is a block diagram of the classification device 11 that inputs a survey target document and extracts opinions using a classification model such as a discrimination function generated from a training example including a positive example attribute evaluation pair input by an operator. The description of the classification device 11 shown in FIG. 2 will be described later.

  FIG. 1 shows a training document storage unit 1 such as a hard disk for storing a training document including labeled correct answer data to which an operator gives a correct example label, and learning based on the training document to create a classification model. A learning device 10 and a classification model storage unit 12 that stores the classification model created by the model creation means 220 of the learning device 10 are shown.

  In the training document storage unit 1 shown in FIG. 1, the operator gives a positive example label at least at a location that truly becomes a pair of the attribute expression and the evaluation expression in the document (for example, tags are attached to the attribute expression and the evaluation expression). The training document including the correct answer data with the label (positive example attribute evaluation pair as the opinion of the writer) is stored.

  The learning device 10 illustrated in FIG. 1 reads out a training document from the training document storage unit 1 and stores a set of attribute expressions stored in the attribute expression storage unit 110 and a set of evaluation expressions stored in the evaluation expression storage unit 120. Attribute extraction pair candidate extraction means 100 that extracts and outputs attribute evaluation pair candidates corresponding to combinations of attribute expressions and evaluation expressions existing in the training document, and attribute evaluation that stores attribute evaluation pair candidates Pair candidate storage means 130.

  Further, the learning device 10 reads out the training document from the training document storage unit 1 and sets the attribute evaluation pair candidate that is the same as the attribute expression and evaluation expression pair that the operator gives a label to the training document as the correct answer. A training case creation means 210 that performs a training process for defining a pair candidate and determining an attribute evaluation pair candidate that is not a positive example attribute evaluation pair candidate as a negative example attribute evaluation pair candidate, and a positive example attribute evaluation pair candidate and a negative example attribute evaluation pair candidate And a model creation means 220 for creating a classification model.

  The above-mentioned positive example means that there is a relationship between the attribute expression of the attribute evaluation pair candidate to be focused on and the evaluation expression, and it is a subjective evaluation. That is, the positive example means a true attribute evaluation pair. Further, the negative example means that there is no relationship between the attribute expression and the evaluation expression of the attribute evaluation pair candidate to be noticed, or it is not a subjective evaluation. That is, a negative example means that it is not a true attribute evaluation pair.

  For example, the training case creation means 210 extracts eight attribute evaluation pair candidates from the training sentences S1 and S2 included in the training document shown in FIG. An example attribute evaluation pair (A1, E1, feature 1) and the other seven negative example attribute evaluation pairs are created. The model creation means 220 inputs these to create a classification model and stores it in the classification model storage unit 12.

  The classification device 11 shown in FIG. 2 uses the learning model created by the learning device 10 to extract attribute evaluation pair candidates from the investigation target document stored in the document storage unit 5, and true the attribute evaluation pair candidates. Or a true attribute evaluation pair (negative example).

  The attribute evaluation pair candidate extraction unit 100 of the classification device 11 extracts attribute evaluation pair candidates from the investigation target document in the same manner as the processing executed by the attribute evaluation pair candidate extraction unit 100 of the learning device 10, and the attribute evaluation pair candidates. Store in the storage unit 130. The classification unit 230 reads out the attribute evaluation pair candidates from the attribute evaluation pair candidate storage unit 130 and classifies them into a positive example attribute evaluation pair or a negative example attribute evaluation pair using the classification model stored in the classification model storage unit 12. Perform the process.

  Here, the attribute expression and evaluation expression pair candidates classified into the positive attribute evaluation pairs are determined to be true attribute evaluation pairs, but the learning device 10 shown in FIG. 1 and the classification device shown in FIG. 11 may cause the following problems.

  In the opinion extraction method based on machine learning using the learning device 10 shown in FIG. 1 and the classification device 11 shown in FIG. 2, there is a problem of whether there is a relationship between attribute expression and evaluation expression, and is it subjective evaluation? Therefore, there is a high possibility that learning in which two problems are mixed will be performed. Therefore, there is a low possibility that the extracted positive example attribute evaluation pair has extracted the opinion of the writer and the extraction performance may not be improved.

  For example, in the example shown in FIG. 1, there are S1 and S2 sentences as training sentences, and the attribute evaluation pair of the GID1 group extracted from S1 includes a positive example attribute evaluation pair, but is extracted from the training sentence S2. The attribute evaluation pair of the GID2 group is composed of only negative example attribute evaluation pairs, and does not include any positive example attribute evaluation pair.

  The negative example attribute evaluation pair extracted from the training sentence S1 is highly likely to be a negative example attribute evaluation pair because the operator specifies that there is no relationship between the attribute expression and the evaluation expression. However, in the training sentence S2, since the operator does not specify the positive example attribute evaluation pair, the negative example attribute evaluation pair extracted by the learning device 10 from the training sentence S2 is not related to the attribute expression and the evaluation expression. It is unclear whether it is determined as a negative example attribute evaluation pair because the operator has a subjective evaluation although it is originally related to the attribute expression and the evaluation expression. Therefore, if learning is performed for all cases using the learning device 10, learning in which the above two problems are mixed is performed, and it can be predicted that this does not improve opinion extraction performance.

  The best mode for carrying out the present invention will be described below with reference to the drawings.

(First embodiment)
FIG. 3 shows a classification model 1 such as a discriminant function Z1 and a classification model 2 such as a discriminant function Z2 created by learning attribute candidate pairs from a training case including a positive example attribute evaluation pair input by an operator. By generating and outputting, the classification to determine in what kind of document the attribute expression and evaluation expression become a normal example attribute evaluation pair, and conversely in which case it does not become a normal example attribute evaluation pair It is a block diagram regarding the process of the learning apparatus 2 which learns the model 1 and the classification model 2. FIG.

  FIG. 21 is a diagram illustrating a survey target document using a classification model 1 such as a discriminant function Z1 and a classification model 2 such as a discriminant function Z2 created by the learning device 2 shown in FIG. It is a block diagram of the classification | category apparatus 7 which extracts a positive example opinion. The description of the classification device 7 shown in FIG. 21 will be described later.

  FIG. 3 shows a training document storage unit 1 such as a hard disk for storing a training document including correct data with a label to which an operator gives a correct example label, and an identification function Z1 and the like by performing learning based on the training document. A learning apparatus 2 for creating a classification model 1 using the classification model 1 and a discrimination function Z2 used, and a classification model 1 for storing a classification model 1 such as a discrimination function Z1 created by the model creation 1 means 150 of the learning apparatus 2 A storage unit 3 and a classification model 2 storage unit 4 that stores the classification model 2 such as the discriminant function Z2 created by the model creation 2 means 190 of the learning device 2 are shown.

  The learning device 2 shown in FIG. 1 reads a training document from the training document storage unit 1 and stores a set of attribute expressions stored in the attribute expression storage unit 110 and a set of evaluation expressions stored in the evaluation expression storage unit 120. Attribute evaluation pair candidate extraction means 100 for extracting and outputting attribute evaluation pair candidates corresponding to a combination of attribute expression and evaluation expression existing in the training document for each training document, and attributes, Attribute evaluation pair candidate storage means 130 for storing evaluation pair candidates.

  Further, the learning device 2 reads out the training document from the training document storage unit 1, and selects the attribute evaluation pair candidate that is the same as the attribute expression / evaluation expression pair that the operator gave a label to the training document as the correct example attribute evaluation. A training case creation 1 means 140 that performs a process of defining an attribute evaluation pair candidate that is not a positive example attribute evaluation pair candidate as a negative example attribute evaluation pair candidate in a group in which a positive example attribute evaluation pair candidate exists, Model creation that creates classification model 1 such as discriminant function Z1 that mainly learns whether or not there is a correct relationship between attribute expression and evaluation expression by inputting example attribute evaluation pair candidate and negative example attribute evaluation pair candidate 1 means 150.

  In addition, the learning device 2 uses the discriminant function Z1 stored in the classification model 1 storage unit 3 to determine the undetermined attribute evaluation in the same group ID that does not belong to the positive example attribute evaluation pair candidate or the negative example attribute evaluation pair candidate. Classification 1 means 160 for performing processing for classifying the pair candidate group into positive example attribute evaluation pair candidates or negative example attribute evaluation pair candidates, and positive example attribute evaluation pair candidates or negative example attribute evaluation pair candidates classified by classification 1 means 160 And a classification 1 means classification result storage unit 170 for storing.

  Further, the learning device 2 reads out the training document from the training document storage unit 1, and selects the attribute evaluation pair candidate that is the same as the attribute expression / evaluation expression pair that the operator gave a label to the training document as the correct example attribute evaluation. A training case creation 2 means 180 that reads out the positive example attribute evaluation pair candidate stored in the classification 1 means classification result storage unit 170 and replaces it with a negative example attribute evaluation pair candidate, By inputting the positive example attribute evaluation pair candidate extracted by the creation 2 means 180 and the replaced negative example attribute evaluation pair candidate, the classification model 2 using the discriminant function Z2 for mainly learning the subjective evaluation from the training document is obtained. Model creation 2 means 190 to create is provided.

  In the training document storage unit 1 shown in FIG. 3, an operator gives a positive example label at least at a location that truly becomes a pair of an attribute expression and an evaluation expression in the document (for example, a tag is attached to the attribute expression or the evaluation expression). The training document including the correct answer data with the label (positive example attribute evaluation pair as the opinion of the writer) is stored.

  FIG. 4 is a diagram illustrating an example of a training document.

  This figure shows an example in which two types of training texts, that is, training text S1 and training text S2, which are difficult to extract simply as opinions, are input and stored. As shown in the figure, the training sentence S1 is “Design does not know the price and support performance, but it is good”, and this training sentence has a dependency relationship “Design is good”. , The attribute expression “design” and the evaluation expression “good” are included as a positive example attribute evaluation pair as an opinion of the writer. Therefore, the operator surrounds the attribute expression “design” with <attribute: 1> and </ attribute> tags (one form of label) and attaches the same id as the attribute expression to the evaluation expression “good” < Evaluation: 1> </ evaluation> tags (one form of label), and the learning device 2 can determine the combination of the attribute expression with the same id and the tag of the evaluation expression as a positive attribute evaluation pair. And stored in the training document storage unit 1.

  One training sentence S2 is “having comfortability, design, engine sound, and quietness, and may be good.” In this training sentence, an attribute evaluation pair to be extracted as an opinion of the writer Does not exist. Therefore, the operator stores the training sentence S2 in the training document storage unit 1 without giving a special label. In addition, the symbols S1 and S2 for identifying the training sentences are given for the explanation of the present invention and need not be given.

  FIG. 5 is a diagram showing a state in which a training document is generally expressed.

  As shown in the figure, attributes (attribute expressions such as “design”, “price”, “support”) in the training document S1 and training sentence S2 shown in FIG. 4 are represented by A1, A2,. It can be expressed as a replacement. Similarly, evaluations in the training sentence (evaluation expressions such as “good”, “cheap”, and “good”) can be expressed as E1,..., En.

  FIG. 6 is a diagram showing a set of attribute expressions stored in the attribute expression storage unit 110, and FIG. 7 is a diagram showing a set of evaluation expressions stored in the evaluation expression storage unit 120.

  As shown in FIG. 6 and FIG. 7, the learning device 2 has “design” = A1, “price” = A2, “support” = A3, “performance” = A4, etc. that are also present in the training document input by the operator. An attribute expression storage unit 110 that stores in advance an attribute expression representing an aspect of a specific target, and an evaluation expression related to the attribute expression, where “good” = E1, “cheap”, “good”, An evaluation expression storage unit 120 that stores in advance Japanese evaluation expressions corresponding to attribute expressions such as “like”, “worst”, and “easy to use” is provided.

  The attribute expression or the evaluation expression stored in the attribute expression storage unit 110 or the evaluation expression storage unit 120 is an attribute expression of correct answer data (correct example attribute evaluation pair) specified in the training document stored in the training document storage unit 1. And a pair of evaluation expressions may be extracted and stored.

  The attribute evaluation pair candidate extraction unit 100 of the learning device 2 reads out the training document from the training document storage unit 1, the attribute expression stored in the attribute expression storage unit 110, and the evaluation stored in the evaluation expression storage unit 120. The processing is performed by referring to the expression and extracting a portion corresponding to the combination of the attribute expression and the evaluation expression existing in the training document. For example, a combination of attributes A1,..., An and evaluations E1,..., En is extracted from the training document and output as attribute expression pair candidates.

  Then, the attribute evaluation pair candidate extraction unit 100 has the value of the feature (in the example shown in FIG. 3, described as feature 1 to feature 8. Details of the feature will be described later) within a predetermined range. A pair of an attribute expression and an evaluation expression ((A1, E1) to (A7, E1) in the example shown in FIG. 3) is set as an “attribute evaluation pair candidate”. And about each attribute evaluation pair candidate, group ID (GID) which shows from which range of the training document it extracted, and the information of a feature are provided.

  Here, as a condition for the attribute evaluation pair candidate to exist within the predetermined range of the training document, for example, the condition “the attribute expression exists in the same sentence as the sentence including the evaluation expression” or “the evaluation expression And the preceding k sentences (where k is a natural number) can be considered, and grouping will be described below with reference to FIGS.

  FIG. 8 is a diagram illustrating an example of a training document stored in the training document storage unit 1. In the example shown in the figure, training sentences S3 to S5 are stored.

  FIG. 9 shows that the attribute evaluation pair candidate extraction unit 100 has a group (GID1) of attribute evaluation pair candidates in which the attribute expression exists in the same sentence as the sentence including the evaluation expression E1, and the same sentence as the sentence including the evaluation expression E2. It is a figure which shows the example of classification classified into the group (GID2) of the attribute evaluation pair candidate in which attribute expression exists.

  In FIG. 10, the attribute evaluation pair candidate extraction unit 100 includes a sentence including the evaluation expression E1, a group of attribute evaluation pair candidates (GID1) in which the attribute expression exists within one sentence in front of the sentence, a sentence including the evaluation expression E2, and It is a figure which shows the example of classification classified into the group (GID2) of the attribute evaluation pair candidate in which attribute expression exists within one sentence ahead.

  FIG. 11 shows a condition that an attribute expression exists in the same sentence as a sentence including an evaluation expression, and a positive example attribute tag (a form of a label) that exists in a place closest to the evaluation expression. In the example, <attribute: 1> A1 </ attribute> corresponds to A1)) is a diagram illustrating a classification example in which the range up to A1 is limited as a classification range.

Next, feature information will be described. The feature information is, for example, information that can be represented by an array, and one or a combination of the following features can be used.
Feature a: Surface character string feature of clause containing attribute expression and evaluation expression b: Part of speech feature of clause containing attribute expression and evaluation expression c: Whether clause containing attribute expression and clause containing evaluation expression are in a dependency relationship Feature d: Distance information between attribute expression and evaluation expression

  FIG. 12 is a diagram illustrating a training sentence S6 and a training sentence S7 stored in the training document storage unit 1.

  Here, a description will be given by taking as an example processing performed when the attribute evaluation pair candidate extraction unit 100 assigns feature information to the attribute evaluation pair candidate (design, good) in the training sentence S6 and the training sentence S7 illustrated in FIG. . First, in order to obtain features, word breaks and phrase breaks are detected by using morphological analysis of the training sentence S6 and the training sentence S7 (see FIG. 13). Next, the dependency relationship between clauses is detected using syntax analysis (see FIG. 13).

  FIG. 13 is a diagram illustrating the results of performing morphological analysis and syntax analysis on the training sentence S6 and the training sentence S7.

  In the example shown in the figure, “+” represents a word break obtained as a result of morphological analysis, and “/” represents a phrase break. Also, the part of speech of the word obtained as a result of the syntax analysis is represented by a character string between “[]”, and the dependency relationship between phrases is represented by “→”.

  FIG. 14 is a diagram illustrating results of extracting features using results of morphological analysis and syntax analysis.

  The feature a will be described with reference to FIG. In the example shown in the figure, the surface character string of the clause including the attribute expression and the evaluation expression corresponds to each word delimited by “+”, and this surface character string is used as “feature a”. If an array of dimensions of all kinds of surface character strings is prepared and each word is present as a candidate candidate feature, 1 (or flag) is used as a binary value of the feature a of each word. Set. If each word does not exist as a target candidate feature, 0 (or a flag) is set as the binary value of the feature a of each word. In general, training sentences contain many words, so this array is a very high-dimensional array. Note that the object of the present invention can be achieved by setting the appearance frequency of a word instead of a binary value.

  The feature b will be described with reference to FIG. Each part of speech between “[]” corresponds to the part of speech of the phrase including the attribute expression and the evaluation expression. An array of dimensions of the number of types of parts of speech is prepared, and when each part of speech exists as a target candidate feature, 1 is set as the binary value of the feature b. If each part of speech does not exist as a target candidate feature, 0 is set as the binary value of the feature b. Also in this case, the object of the present invention can be achieved by substituting the appearance frequency of the part of speech instead of the binary value.

  The feature c will be described with reference to FIG. When a clause including an attribute expression and a clause including an evaluation expression are in a dependency relationship, the clause connected by “→” corresponds to the dependency relationship. If it is determined that there is a dependency relationship between phrases, 1 (or flag) is set as the binary value of the feature c. If it is determined that there is no dependency relationship between phrases, 0 (or a flag) is set as the binary value of the feature c.

  The feature d will be described with reference to FIG. As the distance information between the attribute expression and the evaluation expression, for example, the number of bytes between the attribute expression word and the evaluation expression word is set. Also, the object of the present invention can be achieved by substituting the number of characters instead of the number of bytes. In the above embodiment, positive logic or negative logic may be used for binary values or flags of the feature a, the feature b, and the feature c.

  FIG. 15 is a diagram illustrating combinations of attribute evaluation pair candidates stored in the attribute evaluation pair candidate storage unit 130.

  The attribute evaluation pair candidate storage unit 130 of the learning device 2 illustrated in FIG. 3 stores the attribute evaluation pair candidates extracted by the attribute evaluation pair candidate extraction unit 100. In the example shown in the figure, the set of attribute expressions stored in the attribute expression storage unit 110 and the set of evaluation expressions stored in the evaluation expression storage unit 120 are referred to and stored in the training document storage unit 1. The example which memorize | stored the result of having extracted the attribute evaluation pair candidate from the trained training document is shown. GID1 and GID2 shown in the figure indicate group IDs. Features 1 to 7 are the features of the attribute evaluation pair candidates (may be information including an array). In the grouping shown in the figure, a group ID is given with a predetermined range within the same sentence.

  FIG. 16 shows positive example attribute evaluation pair candidates and negative example attribute evaluation pair candidates (training examples) output by the training case creation 1 means 140 based on the attribute evaluation pair candidates stored in the attribute evaluation pair candidate storage unit 130. FIG.

  The training example creation 1 means 140 of the learning device 2 shown in FIG. 3 is a part of the training document stored in the training document storage unit 1 where a correct example attribute evaluation pair exists (correct answer given by the operator as a correct example). A document including the opinion of the writer, and the attribute evaluation pair candidate identical to the attribute expression / evaluation expression pair extracted from the document) is determined as a positive example attribute evaluation pair candidate. Also, attribute evaluation pair candidates that are not positive attribute evaluation pair candidates (a pair of an attribute expression and an evaluation expression that the operator has not given a label as a positive example) in a GID in which there are positive attribute evaluation pair candidates are negative example attributes. Processing to determine as an evaluation pair candidate is performed.

  In the example shown in the figure, the group attribute evaluation pair candidate of GID2 stored in the attribute evaluation pair candidate storage unit 130 is an attribute evaluation pair candidate extracted from a training sentence that does not include any of the normal example attribute evaluation pairs of the training document. For this reason, the training case creation 1 means 140 does not select a positive example attribute evaluation pair candidate or a negative example attribute evaluation pair candidate.

  The model creation 1 means 150 of the learning device 2 shown in FIG. 3 creates the classification model 1 by inputting the positive example attribute evaluation pair candidate and the negative example attribute evaluation pair candidate determined by the training case creation 1 means 140. . The classification model 1 uses an identification function Z1 that determines a boundary surface to be divided into a positive example attribute evaluation pair and a negative example attribute evaluation pair in the subsequent processing. As the discrimination function Z1, for example, the function shown in the following (Formula 1) can be used.

Z1 = a_1 · x_1 + a_2 · x_2 .... + a_n · x_n + a_0 (Formula 1)

  In the above (Formula 1), for x_1 to x_n, an array element included in the feature of the input attribute evaluation pair candidate can be used. For example, in the example shown in FIG. 14 described above, a value corresponding to the binary value (or flag) of the feature a is obtained, such as the surface character string “design” = x_1, “other than” = x_2, “ga” = x_3. Can be used.

  Also, values corresponding to the binary value of the feature b can be used, such as part of speech “noun” = x_1, “auxiliary verb” = x_2. It is also possible to use a binary value (or flag) indicating the presence / absence of a dependency relationship of the feature c or a value indicating distance information of the feature d for x_1 to x_n.

  A_1 to a_n shown in the above (formula 1) are constants. For a_1 to a_n, a linear discriminant used in statistics, a constant determined using Support Vector Machine, or the like can be substituted. For example, in the linear discriminant method, a positive example attribute evaluation pair candidate and a negative example attribute evaluation pair candidate are divided, and x_1 to x_n are assigned to the discriminant function Z1 to calculate a discrimination score Z1x.

  The plane that best separates the two groups of the positive example attribute evaluation pair candidate and the negative example attribute evaluation pair candidate has the maximum correlation ratio (Sb / St) when St is the total variation and Sb is the interclass variation. It is a plane. Therefore, a_1 to a_n are selected so that (Sb / St) is maximized. The total variation St is a value indicating how much each data is dispersed from the total average Z_all of the discrimination score Z1x, and can be obtained by the following (Equation 2).

St = ΣiΣj (Z_ij−Z_all) ² (Expression 2)

  Here, i is a dummy index representing the type of group (positive example attribute evaluation pair candidate or negative example attribute evaluation pair candidate), and takes a value of 1 or 2 here. J is a dummy index representing input data. The interclass variation Sb is a value indicating how far the positive attribute evaluation pair and the negative attribute evaluation pair are dispersed from the total average Z_all, respectively, and is obtained by the following (Equation 3). Can do.

Sb = Σi {n_i · (Z_i−Z_all) ² } (Formula 3)

  Here, i is a dummy index representing the type of group (positive example attribute evaluation pair candidate or negative example attribute evaluation pair candidate), and takes a value of 1 or 2 here. N_i represents the number of data in the i-th group.

  The classification model 1 storage unit 3 shown in FIG. 3 stores the classification model 1 such as the discriminant function Z1 created by the model creation 1 means 150.

  In the classification 1 means 160 of the learning device 2 shown in FIG. 3, the attribute evaluation pair candidate in the attribute evaluation pair candidate storage unit 130 is first input, the training document is read from the training document storage unit 1, and the operator labels the training document. Of extracting attribute evaluation pair candidates belonging to a group (GID) made up of attribute evaluation pair candidates that do not include any of the same attribute evaluation pair candidates as the positive example attribute evaluation pair to I do. The attribute evaluation pair candidates extracted here are undetermined attribute evaluation pair candidates that do not belong to the positive example attribute evaluation pair candidate and do not belong to the negative example attribute evaluation pair candidate. An extraction example of undetermined attribute evaluation pair candidates is shown in FIG. As shown in the figure, the classification 1 means 160 extracts undetermined attribute evaluation pair candidates belonging to GID2.

  FIG. 17 is a diagram showing undetermined attribute evaluation pair candidates extracted by the classification 1 means as described above.

  Next, the classification 1 means 160 uses the discriminant function Z1 stored in the classification model 1 storage unit 3 to determine a group of undetermined attribute evaluation pairs belonging to a group (GID) that does not include any positive attribute evaluation pairs. Is classified into a positive example attribute evaluation pair candidate or a negative example attribute evaluation pair candidate. For example, the classification 1 means 160 substitutes the feature of the attribute evaluation pair candidate for x_1 to x_n of the discrimination function Z1 recorded in the classification model 1 storage unit 3, and calculates the value of the discrimination score Z1x. The discriminant function Z1 represents a boundary surface that divides an undetermined attribute evaluation pair candidate into a positive example attribute evaluation pair candidate and a negative example attribute evaluation pair candidate, so that the discrimination score Z1x is calculated to be a positive value or negative Can be classified into positive example attribute evaluation pair candidates or negative example attribute evaluation pair candidates.

  FIG. 18 is a diagram showing positive example attribute evaluation pair candidates and negative example attribute evaluation pair candidates classified by the classification 1 means.

  The classification 1 means classification result storage unit 170 of the learning device 2 shown in FIG. 3 stores the classification results of the classification 1 means 160 as shown in FIG.

  The training case creation 2 means 180 of the learning device 2 shown in FIG. 3 reads the training document from the training document storage unit 1 and is the same as the pair of the attribute expression and the evaluation expression that are regarded as correct answers by the operator labeling the training document. Are extracted as positive example attribute evaluation pair candidates, and the positive example attribute evaluation pair candidates stored in the classification 1 means classification result storage unit 170 are read and replaced with negative example attribute evaluation pair candidates. .

  FIG. 19 is a diagram showing the positive example attribute evaluation pair candidates extracted by the training case creation 2 means 180 and the replaced negative example attribute evaluation pair candidates. As shown in the figure, the positive example attribute evaluation pair candidate extracted by the training case creation 2 means 180 is (A1, E1, feature 1) of GID1, and is given as a negative example by the classification 1 means 160. (A1, E1, feature 6) determined to be an example attribute evaluation pair candidate.

  The model creation 2 means 190 of the learning apparatus 2 shown in FIG. 3 inputs the positive example attribute evaluation pair candidate extracted by the training case creation 2 means 180 and the replaced negative example attribute evaluation pair candidate, and uses the discriminant function Z2. The classification model 2 that was found is created. The method of creating the classification model 2 using the discriminant function Z2 is the same as that of the model creation 1 means 150, and thus description thereof is omitted here.

  The classification model 2 storage unit 4 of the learning device 2 shown in FIG. 3 stores the classification model 2 of the discriminant function Z2 created by the model creation 2 means 190.

  FIG. 20 is a flowchart of processing for extracting the classification model 1 such as the discrimination function Z1 and the classification model 2 such as the discrimination function Z2 using the learning device 2. The processing steps from S100 to S190 shown in the figure correspond to the processing in each means of the learning device 2 in FIG.

  FIG. 21 shows a case where a learning device 2 uses a classification model 1 such as a discriminant function Z1 and a classification model 2 such as a discriminant function Z2 created from a training case including a positive example attribute evaluation pair candidate. It is a block diagram of the classification | category apparatus 7 which performs the process which extracts a pair as a positive opinion.

  A set of attribute expressions stored in the attribute expression storage unit 110 of the classification device 7 illustrated in FIG. 21 and a set of evaluation expressions stored in the evaluation expression storage unit 120 are attributes stored in the attribute expression storage unit 110 illustrated in FIG. Expression information and information similar to the information stored in the evaluation expression storage unit 120 can be used.

  The classification device 7 shown in the figure reads a survey target document from the document storage unit 5 and stores a set of attribute expressions stored in the attribute expression storage unit 110 and a set of evaluation expressions stored in the evaluation expression storage unit 120. The attribute evaluation pair candidate extraction means 200 for extracting and outputting attribute evaluation pair candidates corresponding to the combination of the attribute expression and the evaluation expression existing in the investigation target document, and the attribute evaluation pair candidate are stored. Attribute evaluation pair candidate storage means 230.

  Further, the classification device 7 inputs the attribute evaluation pair candidate and the classification model 1 such as the discriminant function Z1 stored in the classification model 1 storage unit 3, and selects the attribute evaluation pair candidate in the same group ID as the normal example attribute evaluation. Classification 1 means 260 for performing processing for classifying into pair candidates or negative example attribute evaluation pair candidates, and classification 1 means classification result storage for storing positive example attribute evaluation pair candidates or negative example attribute evaluation pair candidates classified by classification 1 means 260 Part 270.

  Further, the classification device 7 uses the classification model 2 such as the discriminant function Z2 stored in the classification model 2 storage unit 4 to classify the attribute evaluation pair candidates into positive example attribute evaluation pairs or negative example attribute evaluation pairs. A classification 2 means 290 to perform, and a classification 2 means classification result storage unit 6 that stores positive example attribute evaluation pairs or negative example attribute evaluation pairs classified by the classification 2 means 290 are provided.

  The document storage unit 5 illustrated in FIG. 21 is a storage unit such as a hard disk for storing the investigation target document. Survey target documents include Web bulletin boards such as blogs, E-mails, questionnaire results, inquiry data for company inquiry windows, reports, and other documents. Unlike the training documents stored in the training document storage unit 1 illustrated in FIG. 3, these survey target documents are documents that are not labeled with positive attribute evaluation pairs. For example, the case where the investigation target sentence S10 as shown in FIG. 22 is stored in the document storage unit 5 will be described below.

  Further, the processing performed by the attribute evaluation pair candidate extraction unit 200 illustrated in FIG. 21 and the processing performed by the attribute evaluation pair candidate storage unit 230 include the attribute evaluation pair candidate extraction unit 100 of the learning device 2 illustrated in FIG. Since it is the same as the process which the attribute evaluation pair candidate memory | storage part 130 implements, description of a function is abbreviate | omitted. For example, when the attribute evaluation pair candidate extraction process is performed on the investigation target sentence S10 illustrated in FIG. 22, the attribute evaluation pair candidate storage unit 230 extracts and stores the attribute evaluation pair candidates illustrated in FIG.

  FIG. 23 is a diagram illustrating an example of attribute evaluation pair candidates stored in the attribute evaluation pair candidate storage unit 230. In the example shown in the figure, (A1, E1, feature 1) attribute evaluation pair candidates are extracted and stored.

  The classification 1 means 260 of the classification device 7 shown in FIG. 21 includes a classification model such as an attribute evaluation pair candidate stored in the attribute evaluation pair candidate storage unit 230 and a discrimination function Z1 stored in the classification model 1 storage unit 3. 1 is input, and the attribute evaluation pair candidate is classified into a positive example attribute evaluation pair candidate or a negative example attribute evaluation pair candidate using the discriminant function Z1.

  For example, the classification 1 means 260 first calculates the discrimination score Z1x by substituting the features of the attribute evaluation pair candidates for x_1 to x_n of the discrimination function Z1 recorded in the classification model 1 storage unit 3. The discriminant function Z1 represents a boundary surface that separates the positive example attribute evaluation pair candidate and the negative example attribute evaluation pair candidate, so that the positive example depends on whether the value of the discrimination score Z1x is a positive value or a negative value It is possible to classify into attribute evaluation pair candidates or negative example attribute evaluation pair candidates. The classified attribute evaluation pair candidates are stored in the classification 1 means classification result storage unit 270.

  FIG. 24 is a diagram illustrating attribute evaluation pair candidates stored in the classification 1 means classification result storage unit 270.

  The classification 1 means classification result storage unit 270 of the classification device 7 shown in FIG. 21 stores positive example attribute evaluation pair candidates or negative example attribute evaluation pair candidates classified by the classification 1 means 260 (in the example shown in FIG. 21, (A1, E1 , Feature 1) = positive example attribute evaluation pair candidate).

  The classification 2 means 290 shown in FIG. 21 uses the classification model 2 such as the discriminant function Z2 stored in the classification model 2 storage unit 4 as the attribute evaluation pair candidate determined by the classification 1 means 260 as the positive example attribute evaluation pair candidate. Is used to classify into a positive example attribute evaluation pair or a negative example attribute evaluation pair.

  Since the attribute evaluation pair classified into the positive example attribute evaluation pair by the classification 2 means 290 is considered as the opinion of the writer of the investigation target document, the classification 2 means 290 determines that it is a true attribute evaluation pair. Here, the classification 2 means 290 calculates the discrimination score Z2x by substituting the feature of the attribute evaluation pair candidate into x_1 to x_n of the discrimination function Z2 recorded in the classification model 2 storage unit 4. The discriminant function Z2 represents a boundary surface that separates the positive example attribute evaluation pair and the negative example attribute evaluation pair, and therefore can be classified into a positive example attribute evaluation pair or a negative example attribute evaluation pair depending on whether the value of the discrimination score Z2x is positive or negative. is there. The positive example attribute evaluation pair or the negative example attribute evaluation pair classified by the classification 2 means 290 is stored in the classification 2 means classification result storage unit 6.

  FIG. 25 is a diagram showing a positive example attribute evaluation pair or a negative example attribute evaluation pair stored in the classification 2 means classification result storage unit 6. In the example shown in the figure, (A1, E1, feature 1) = positive example attribute evaluation pair) is stored. The positive example attribute evaluation pair candidates determined as the positive example attribute evaluation pair by the classification 2 means 290 are extracted and stored as the true attribute evaluation pair, that is, the opinion of the writer of the document to be investigated.

  FIG. 26 is a flowchart of processing for extracting the opinion of the writer from the investigation target document using the classification device 7. The processing steps from S200 to S290 shown in the figure correspond to the processing in each means of the classification device 7 in FIG.

(Second Embodiment)
FIG. 27 shows a classification model 3 such as an identification function Z3 and a classification model 4 such as an identification function Z4 created by learning attribute evaluation pair candidate classification from a training example including a positive example attribute evaluation pair input by an operator. By generating and outputting, the classification to determine in what kind of document the attribute expression and evaluation expression become a normal example attribute evaluation pair, and conversely in which case it does not become a normal example attribute evaluation pair It is a block diagram regarding the process of the learning apparatus 8 which makes the model 3 and the classification | category model 4 learn.

  FIG. 31 is a diagram illustrating a survey target document using a classification model 3 such as a discriminant function Z3 and a classification model 4 such as a discriminant function Z4 created from a training case including a positive example attribute evaluation pair candidate by the learning device 8 shown in FIG. It is a block diagram of the classification | category apparatus 9 which extracts a positive example opinion. The description of the classification device 9 shown in FIG. 31 will be described later.

  FIG. 27 shows a training document storage unit 1 such as a hard disk for storing a training document including labeled correct answer data to which an operator has given a correct example label, and learning function based on the training document to obtain an identification function Z3 and the like. A learning model 8 that creates a classification model 4 using the classification model 3 and the discrimination function Z4 used, and a classification model 1 that stores the classification model 3 such as the discrimination function Z3 created by the model creation 1 means 350 of the learning device 8 A storage unit 3 and a classification model 2 storage unit 4 that stores the classification model 4 of the discriminant function Z4 created by the model creation 2 means 190 of the learning device 8 are shown.

  The learning device 8 shown in FIG. 27 reads a training document from the training document storage unit 1 and stores a set of attribute expressions stored in the attribute expression storage unit 110 and a set of evaluation expressions stored in the evaluation expression storage unit 120. Attribute extraction pair candidate extraction means 100 that extracts and outputs attribute evaluation pair candidates corresponding to combinations of attribute expressions and evaluation expressions existing in the training document, and attribute evaluation that stores attribute evaluation pair candidates Pair candidate storage means 130.

  As the training document stored in the training document storage unit 1 of the learning device 8 illustrated in FIG. 27, a training document similar to the training document stored in the training document storage unit 1 illustrated in FIG. 3 can be used. Also, the set of attribute expressions stored in the attribute expression storage unit 110 and the set of evaluation expressions stored in the evaluation expression storage unit 120 are information on attribute expressions stored in the attribute expression storage unit 110 shown in FIG. Information similar to the information stored in the expression storage unit 120 can be used.

  Further, the learning device 8 reads out the training document from the training document storage unit 1 and includes one or more attribute evaluation pair candidates that are the same as the pair of the attribute expression and the evaluation expression that are regarded as correct answers by the operator labeling the training document. Extract attribute evaluation pair candidates for group ID and generate attribute evaluation comparison pair candidates based on the features and positional relationships of the two attribute expressions in the two attribute evaluation pair candidates. When the attribute expression existing in (front) matches the attribute expression of the positive example attribute evaluation pair in the training document, an attribute evaluation comparison pair in which the Left label is assigned to the attribute evaluation comparison pair candidate is generated, When the attribute expression existing on the right side (rear) from the beginning in the training document matches the attribute expression of the positive example attribute evaluation pair in the training document, a right label is assigned to the attribute evaluation comparison pair candidate Comprising a training set created 1 means 340 for performing the attribute evaluation process generates a comparison pairs.

  Further, the learning device 8 includes a model creation 1 means 350 for creating a classification model 3 such as a discriminant function Z3 that mainly learns whether an attribute expression and an evaluation expression have a correct relationship by inputting an attribute evaluation comparison pair. Prepare.

  Further, the learning device 8 creates an attribute evaluation comparison pair from two sets of undetermined attribute evaluation pair candidates in the same group ID, and the discrimination function Z3 stored in the classification model 1 storage unit 3 and the undetermined attribute evaluation pair. The candidate 1 is used to compare in order to the tournament method, and the classification 1 means 360 for classifying the positive example attribute evaluation pair candidate and the negative example attribute evaluation pair candidate for each same group ID, and the positive classified by the classification 1 means 360 A classification 1 means classification result storage unit 170 that stores example attribute evaluation pair candidates or negative example attribute evaluation pair candidates is provided.

  Further, the learning device 8 reads out the training document from the training document storage unit 1, and selects the attribute evaluation pair candidate that is the same as the pair of the attribute expression and the evaluation expression as the correct answer by which the operator has attached the label to the training document. A training case creation 2 means 180 that reads out the positive example attribute evaluation pair candidate stored in the classification 1 means classification result storage unit 170 and replaces it with a negative example attribute evaluation pair candidate, The classification model 4 using the identification function Z4 or the like for mainly learning the subjective evaluation from the training document by inputting the positive example attribute evaluation pair candidate extracted by the creation 2 means 180 and the replaced negative example attribute evaluation pair candidate. Model creation 2 means 190 for creating

  FIG. 28 is a diagram showing attribute evaluation comparison pairs (training examples) created by the training case creation 1 means 340 using the attribute evaluation pair candidates stored in the attribute evaluation pair candidate storage unit 130.

  The training example creation 1 means 340 of the learning device 8 shown in FIG. 27 first includes a place where a positive example attribute evaluation pair exists in the training document stored in the training document storage unit 1 (the operator gave a label as a positive example) Processing to extract attribute evaluation pair candidates having a group ID including at least one attribute evaluation pair candidate identical to the attribute expression / evaluation expression pair extracted from the correct document and including the opinion of the writer. Do. In the example shown in FIG. 27, attribute evaluation pair candidates belonging to the group of GID1 are extracted.

  Next, the training case creation 1 means 340, among the extracted attribute evaluation pair candidates, any two attribute expressions including a pair of attribute expression and evaluation expression that match the positive attribute evaluation pair, and one evaluation expression pair. A process of generating an attribute evaluation comparison pair candidate in which is combined is performed. In the example shown in FIG. 27, the group attribute evaluation pair candidate of GID2 stored in the attribute evaluation pair candidate storage unit 130 is an attribute evaluation extracted from a training sentence that does not include any of the normal example attribute evaluation pairs of the training document. Since it is a pair candidate, the training example creation 1 means 340 does not select an attribute evaluation pair candidate belonging to the group of GID2 as an attribute evaluation comparison pair.

  Next, the training example creation 1 means 340, based on the features and positional relationships of the two attribute expressions in the attribute evaluation comparison pair candidates, the attribute expression existing from the beginning to the left (front) in the training document is the training document. If it matches the attribute expression of the positive example attribute evaluation pair, processing for generating an attribute evaluation comparison pair in which the Left label is assigned to the attribute evaluation comparison pair candidate is performed. The attribute evaluation comparison pair to which the Left label is assigned indicates that the left attribute expression has a feature closer to the positive example attribute evaluation pair than the right attribute expression. For example, in the example shown in FIG. 28, the attribute expression “A1” is an attribute expression having a feature closer to the positive example attribute evaluation pair than the attribute expressions “A2”, “A3”, and “A4”. Show.

  Similarly, the training example creation 1 means 340 determines that the attribute expression existing from the beginning to the right (back) in the training document is based on the features and positional relationship between the two attribute expressions in the attribute evaluation comparison pair candidate. If the attribute expression of the positive example attribute evaluation pair in the document matches, processing for generating an attribute evaluation comparison pair in which a Right label is assigned to the attribute evaluation comparison pair candidate is performed. In the attribute evaluation comparison pair to which the Right label is assigned, the right side attribute expression has a feature closer to the positive example attribute evaluation pair than the left side attribute expression.

  For example, when the attribute evaluation pair candidate shown in FIG. 15 is stored in the attribute evaluation pair candidate storage unit 130, GID1 including the attribute evaluation pair candidate that matches the positive example attribute evaluation pair in the training document storage unit 1 Attribute evaluation pair candidates that belong to the group are extracted, and an attribute evaluation comparison pair (for example, FIG. 28) that combines any two attribute evaluation pair candidates including (A1, E1, feature 1) that match the positive example attribute evaluation pair. Three attribute evaluation comparison pairs) are created.

  In the example of the three attribute evaluation comparison pairs shown in FIG. 28, since the attribute expression “A1” on the left side from the top of the document matches the positive example attribute evaluation pair in the training document, “Left” A label is attached. The label “Left” or “Right” may be any type of label as long as it is a label for information indicating the left side and the right side.

  The model creation 1 means 350 of the learning device 8 shown in FIG. 27 inputs the attribute evaluation comparison pair created by the training case creation 1 means 340 and creates the classification model 3. The classification model 3 is created by using the discriminant function Z3 for determining the boundary surface to be divided into Left and Right in the subsequent processing, which is the same as the processing executed by the classification model 1 creation means 150 of the learning device 2 shown in FIG. Use. As the discriminant function Z3, for example, the discriminant function Z1 shown in the above (Formula 1) can be used. The feature of the attribute evaluation comparison pair may be a combination of the two attribute evaluation pair candidate feature sequences of the combination source. The created classification model 3 such as the discriminant function Z3 is stored in the classification model 1 storage unit 3.

  In the classification 1 means 360 of the learning device 8 shown in FIG. 27, first, the attribute evaluation pair candidate is input from the attribute evaluation pair candidate storage unit 130, the training document is read from the training document storage unit 1, and the operator labels the training document. Of extracting attribute evaluation pair candidates belonging to a group (GID) made up of attribute evaluation pair candidates that do not include any of the same attribute evaluation pair candidates as the positive example attribute evaluation pair to I do. The attribute evaluation pair candidates extracted here are undetermined attribute evaluation pair candidates as shown in FIG. Although the undetermined attribute evaluation pair candidates shown in FIG. 17 show only the attribute evaluation pair candidates belonging to a single group (GID2), a plurality of groups may be extracted.

  Next, the classification 1 means 360 generates an attribute evaluation comparison pair based on two attribute expression features in two sets of undetermined attribute evaluation pair candidates. Next, the classification 1 means 360 uses the discriminant function Z3 stored in the classification model 1 storage unit 3 to compare the generated attribute evaluation comparison pairs and their features in order to the tournament method, and for each group ID. One attribute evaluation pair candidate that finally remains in is set as a positive example attribute evaluation pair candidate, and attribute evaluation pair candidates other than the positive example attribute evaluation pair candidate are set as negative example attribute evaluation pair candidates.

  For example, the classification 1 means 360 substitutes the feature of the attribute evaluation pair candidate for x_1 to x_n of the discrimination function Z3 recorded in the classification model 1 storage unit 3, and calculates the value of the discrimination score Z3x. The discriminant function Z3 is a function that determines a boundary surface that divides the attribute evaluation pair candidate into Left and Right, so that the discrimination score Z3x is calculated as a positive value or a negative value. Thus, processing for generating an attribute evaluation comparison pair is performed by dividing into Left and Right.

  Next, processing for sequentially comparing attribute evaluation pair candidates using the tournament method will be described. First, the classification 1 means 360 compares two attribute evaluation pair candidates including the attribute expression closest to the right side of the document. That is, an attribute evaluation comparison pair obtained by combining these two attribute evaluation pair candidates is created, and is classified into Left or Right using the classification model 3 of the model 1 storage unit 3. In the case of Left, the left attribute evaluation pair A candidate is left, and in the case of Right, a process of leaving a right attribute evaluation pair candidate is performed. In subsequent comparisons, the process of sequentially comparing the two attribute evaluation pair candidates including the rightmost attribute expression among the attribute evaluation comparison pairs that have not been compared with the remaining attribute evaluation pair candidates is repeated.

  Whether the classification 1 means 360 is classified as a positive example attribute evaluation pair candidate or a negative example attribute evaluation pair candidate depends on the feature of the attribute evaluation candidate pair, but the attribute evaluation pair candidate shown in FIG. When using and classifying, it classifies based on the attribute evaluation pair candidate (GID2 attribute evaluation pair candidate that was not used in the training example creation 1 means) shown in FIG. As a result of the classification, a case where the classification is performed as shown in FIG. 18 and stored in the classification 1 means classification result storage unit 170 will be described.

  FIG. 29 is a diagram showing how the classification 1 means 360 performs classification using the tournament method.

  First, the classification 1 means 360 compares the attribute evaluation comparison pair ((A7, E1, feature 8), (A6, E1, feature 7)) that is closest to the right side of the document. A6, A7, E1, feature 7, feature 8) are created. Suppose that this attribute evaluation comparison pair is classified as Left when the discrimination score Z3x is obtained using the classification model 3. In this case, (A6, E1, feature 7) remains.

  Next, it is assumed that (A6, E1, feature 7) and (A1, E1, feature 6) are compared and similarly classified into Left using the classification model 3. In this case, (A1, E1, feature 6) remains. Finally, (A1, E1, feature 6) and (A5, E1, feature 5) are compared, and it is assumed that classification is performed using the classification model 3. In this case, the positive example attribute evaluation pair candidate (A1, E1, feature 6) remains. As described above, the classification 1 means 360 performs processing for classifying the attribute evaluation pair candidates into positive example attribute evaluation pair candidates and negative example attribute evaluation pair candidates.

  The processing executed by the training case creation 2 means 180 and the model creation 2 means 190 of the learning device 8 shown in FIG. 27 is the same as the processing executed by the learning device 2 shown in FIG. Omitted.

  FIG. 30 is a flowchart of the process of extracting the classification model 3 such as the discrimination function Z3 and the classification model 4 such as the discrimination function Z4 using the learning device 8. The processing steps from S100 to S190 shown in the figure correspond to the processing in each means of the learning device 8 in FIG.

FIG. 31 shows a classification model 3 such as a discriminant function Z3 and a discriminant function Z created by the learning device 8 from training examples including positive example attribute evaluation pair candidates.
It is a block diagram of the classification | category apparatus 9 which performs the process which extracts a positive example attribute evaluation pair as a positive example opinion from an investigation object document using classification models 4, such as 4. FIG.

  A set of attribute expressions stored in the attribute expression storage unit 110 of the classification device 9 illustrated in FIG. 31 and a set of evaluation expressions stored in the evaluation expression storage unit 120 are attributes stored in the attribute expression storage unit 110 illustrated in FIG. Expression information and information similar to the information stored in the evaluation expression storage unit 120 can be used.

  The classification device 9 shown in FIG. 31 reads a survey target document from the document storage unit 5 and stores a set of attribute expressions stored in the attribute expression storage unit 110 and a set of evaluation expressions stored in the evaluation expression storage unit 120. The attribute evaluation pair candidate extraction means 200 for extracting and outputting attribute evaluation pair candidates corresponding to the combination of the attribute expression and the evaluation expression existing in the investigation target document, and the attribute evaluation pair candidate are stored. Attribute evaluation pair candidate storage means 230.

  The classification device 9 shown in FIG. 31 creates an attribute evaluation comparison pair from two groups of attribute evaluation pair candidates in the same group ID stored in the attribute evaluation pair candidate storage unit 230, and the classification model 1 storage unit 3 Are compared in order to the tournament method using the classification model 3 such as the discriminant function Z3 and the attribute evaluation comparison pair stored in the table, and a positive example attribute evaluation pair candidate and a negative example attribute evaluation pair candidate for each group ID A classification 1 means 460 that classifies the data, and a classification 1 means classification result storage unit 270 that stores positive example attribute evaluation pair candidates or negative example attribute evaluation pair candidates classified by the classification 1 means 460.

  Further, the classification device 9 uses the classification model 4 such as the discriminant function Z4 stored in the classification model 2 storage unit 4 to classify the attribute evaluation pair candidates into positive example attribute evaluation pairs or negative example attribute evaluation pairs. A classification 2 means 290 to perform, and a classification 2 means classification result storage unit 6 that stores positive example attribute evaluation pairs or negative example attribute evaluation pairs classified by the classification 2 means 290 are provided.

  The document storage unit 5 shown in FIG. 31 is a storage unit such as a hard disk for storing the investigation target document. Survey target documents include Web bulletin boards such as blogs, E-mails, questionnaire results, inquiry data for company inquiry windows, reports, and other documents. Unlike the training documents stored in the training document storage unit 1 illustrated in FIG. 27, these survey target documents are documents that are not labeled with positive attribute evaluation pairs.

  Further, the processing executed by the attribute evaluation pair candidate extraction unit 200 shown in FIG. 31 and the processing executed by the attribute evaluation pair candidate storage unit 230 are the same as the attribute evaluation pair candidate extraction unit 100 of the learning device 2 shown in FIG. Since the processing is the same as that performed by the attribute evaluation pair candidate storage unit 130, description of the function is omitted. For example, when the attribute evaluation pair candidate extraction process is performed on the investigation target sentence S10 illustrated in FIG. 22, the attribute evaluation pair candidate illustrated in FIG. 23 is extracted and stored in the attribute evaluation pair candidate storage unit 230.

  The classification 1 means 460 of the classification device 9 shown in FIG. 31 first creates an attribute evaluation comparison pair from two attribute evaluation pair candidates in the same group ID stored in the attribute evaluation pair candidate storage unit 230.

  Next, the classification 1 means 460 uses the classification model 3 such as the discriminant function Z3 stored in the classification model 1 storage unit 3 and the attribute evaluation pair candidates to sequentially compare the tournament method, and for each group ID. One attribute evaluation pair candidate remaining finally is classified as a positive example attribute evaluation pair candidate. Further, the classification 1 means 460 performs processing for classifying attribute evaluation pair candidates that are not regarded as positive example attribute evaluation pair candidates as negative example attribute evaluation pair candidates. For the classification process using the tournament method, the same process as the process performed by the classification 1 means 360 of the learning device 8 can be used. Whether the attribute evaluation pair candidate is classified as a positive example attribute evaluation pair candidate or a negative example attribute evaluation pair candidate by the classification 1 means 460 depends on the feature of the attribute evaluation pair candidate. For example, when the attribute evaluation pair candidate shown in FIG. 23 is input and classified, it is classified into the positive example attribute evaluation pair candidate shown in FIG. 24 and stored in the classification 1 means classification result storage unit 270.

  The classification 2 means 290 of the classification device 9 shown in FIG. 31 uses the identification function Z4 stored in the classification model 2 storage unit 4 as the attribute evaluation pair candidate determined as the positive example attribute evaluation pair candidate by the classification 1 means 460. The classification model 4 is used to classify into a positive example attribute evaluation pair or a negative example attribute evaluation pair.

  Since the attribute evaluation pair candidates classified into the positive example attribute evaluation pairs by the classification 2 means 290 are considered to be opinions of the writers of the document to be investigated, the classification 2 means 290 determines that they are true attribute evaluation pairs. The processing executed by the classification 2 means 290 is the same processing as the processing executed by the classification 2 means 290 shown in FIG. The positive example attribute evaluation pair or the negative example attribute evaluation pair classified by the classification 2 means 290 is stored in the classification 2 means classification result storage unit 6.

  FIG. 25 shows positive example attribute evaluation pairs or negative example attribute evaluation pairs stored in the classification 2 means classification result storage unit 6. The positive example attribute evaluation pair candidates determined as the positive example attribute evaluation pair by the classification 2 means 290 are extracted and stored as the true attribute evaluation pair, that is, the opinion of the writer of the document to be investigated.

  FIG. 32 is a flowchart of the process of extracting the writer's opinion from the survey target document using the classification device 9. The processing steps from S200 to S290 shown in the figure correspond to the processing in each means of the classification device 9 in FIG.

  By using the opinion extraction learning apparatus and classification apparatus according to the present invention, it is possible to realize highly accurate opinion extraction that effectively uses a training document input by an operator. The reason is that the classification model 1 or the classification model 3 created by extracting negative examples from the training document and selectively using them so that the relationship between attribute expressions and evaluation expressions can be learned mainly, and subjective evaluation This is because the classification model 2 or the classification model 4 created by selectively using a negative example so as to learn the sex is used.

  The negative example of the model creation 1 means 150 of the learning device 2 that creates the classification model 1 (or the model creation 1 means 350 of the learning device 8 that creates the classification model 3) is a negative example of a group ID including one or more positive examples. Only. A negative example of a group including one or more positive examples is a subjective evaluation, but there is a high possibility that the attribute expression and the evaluation expression are not related. Therefore, the model creation 1 means 150 of the learning device 2 (or the model creation 1 means 350 of the learning device 8) can mainly learn whether there is a correct relationship between the attribute expression and the evaluation expression from the training document. However, it is difficult to learn subjective evaluation from training documents.

  On the other hand, the negative example of the model 2 creation means 190 of the learning device 2 that creates the classification model 2 (or the model creation 2 means 190 of the learning device 8 that creates the classification model 3) is the classification 1 means 160 of the learning device 2 (or Only negative examples erroneously determined as positive examples by the classification 1 means 360) of the learning device 8 are shown. Although it is determined as a positive example by the classification 1 means 160 of the learning device 2 (or the classification 1 means 360 of the learning device 8), there is actually a phenomenon that it is a negative example that does not represent the opinion of the writer of the survey target document. If it has occurred, there is a certain relationship between the attribute expression and the evaluation expression, but there is a high possibility that it is not the subjective evaluation of the writer.

  The model 2 creation means 190 of the learning device 2 according to the present invention (or the model creation 2 means 190 of the learning device 8) can mainly learn subjective evaluation based on the training document. In this way, classification model 1 (or classification model 3) capable of mainly learning whether or not there is a correct relationship between the attribute expression and the evaluation expression from the training document, and mainly learning the subjective evaluation from the training document. By selectively using the negative example of the classification model 2 (or classification model 4) for the purpose, the attribute evaluation pair selected as having the correct relationship only in the classification model 1 (or classification model 3) It is possible to correct the judgment by the classification model 1 using the classification model 2 (or the classification model 3) having learned the evaluation, and improve the opinion extraction accuracy.

  According to the present invention, it is possible to automatically extract a writer's opinion more accurately and automatically from a web bulletin board such as blog or the like, an E-mail, a report document, or other documents that are not intended to describe an opinion itself. As a result, it is possible to improve the efficiency of marketing activities such as a preliminary survey before purchasing a product and a market survey of a company, and to provide a product more in line with the user's wishes at an early stage.

It is a block diagram which shows the structure of the learning apparatus 10 which learns attribute evaluation pair candidate classification | category using a training example, and produces classification models, such as a discriminant function. It is a block diagram which shows the structure of the classification | category apparatus 11 which inputs an investigation object document and extracts an opinion using classification models, such as a discriminant function produced | generated from the training example. It is a block diagram which shows the structure of the learning apparatus of 1st Embodiment which concerns on this invention. It is a figure which shows the example of a training document. It is a figure which shows the state which represented the training document generally. It is a figure which shows the set of the attribute expression which the attribute expression memory | storage part has memorize | stored. It is a figure which shows the set of the evaluation expression which the evaluation expression memory | storage part has memorize | stored. It is a figure which shows an example of the training document memorize | stored in the training document memory | storage part. A group of attribute evaluation pair candidates (GID1) in which the attribute expression exists in the same sentence as the sentence including the evaluation expression E1, and a group of attribute evaluation pair candidates in which the attribute expression exists in the same sentence as the sentence including the evaluation expression E2 ( It is a figure which shows the example of classification classified into GID2). Attribute evaluation pair candidate group (GID1) in which an attribute expression exists within the sentence including the evaluation expression E1 and the preceding sentence, and an attribute evaluation pair in which the attribute expression exists within the sentence including the evaluation expression E2 and the preceding sentence It is a figure which shows the example of classification classified into candidate group (GID2). A classification example in which the condition that the attribute expression exists in the same sentence as the sentence containing the evaluation expression and A1 to which the attribute tag of the positive example existing in the place closest to the evaluation expression is assigned as the classification range FIG. It is a figure which illustrates training sentences S6 and S7 memorized by training document storage part 1. It is a figure which illustrates the result of having implemented morphological analysis and syntactic analysis about training sentences S6 and S7. It is a figure which illustrates the result of having extracted the feature using the result of having performed morphological analysis and syntax analysis. It is a figure which shows the combination of the attribute evaluation pair candidate which an attribute evaluation pair candidate memory | storage part memorize | stores. It is a figure which shows the positive example attribute evaluation pair candidate and negative example attribute evaluation pair candidate which a training example preparation 1 means outputs. It is a figure which shows the undetermined attribute evaluation pair candidate extracted by the classification 1 means. It is a figure which shows the positive example attribute evaluation pair candidate and negative example attribute evaluation pair candidate which the classification 1 means classified. It is a figure which shows the positive example attribute evaluation pair candidate extracted by the training example preparation 2 means, and the replaced negative example attribute evaluation pair candidate. It is a flowchart of the process which extracts the classification model 1 and the classification model 2 using the learning apparatus 2. FIG. It is a block diagram which shows the structure of the classification | category apparatus of 1st Example which concerns on this invention. It is a figure which shows the content of investigation object sentence S10. It is a figure which shows the example of the attribute evaluation pair candidate which an attribute evaluation pair candidate memory | storage part memorize | stores. It is a figure which shows the attribute evaluation pair candidate memorize | stored in the classification 1 means classification result storage part. It is a figure which shows the positive example attribute evaluation pair or negative example attribute evaluation pair memorize | stored in the classification 2 means classification result storage part 6. FIG. 10 is a flowchart of processing for extracting a writer's opinion from a survey target document using the classification device 7. It is a block diagram which shows the structure of the learning apparatus of 2nd Embodiment which concerns on this invention. It is a figure which shows the attribute evaluation comparison pair which the training example preparation 1 means created. It is a figure which shows a mode that a classification | category 1 means classifies using a tournament system. It is a flowchart of the process which extracts the classification model 3 and the classification model 4 using the learning apparatus 8. It is a block diagram which shows the structure of the classification device of 2nd Embodiment which concerns on this invention. 10 is a flowchart of processing for extracting a writer's opinion from an investigation target document using the classification device 9;

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Training document memory | storage part 2, 8, 10 Learning apparatus 3 Classification model 1 memory | storage part 4 Classification model 2 memory | storage part 5 Document memory | storage part 6 Classification 2 means Classification result memory | storage part 7, 9, 11 Classification apparatus 12 Classification model memory | storage part 13 Classification Means classification result storage unit 100, 200 Attribute evaluation pair candidate extraction unit 110 Attribute expression storage unit 120 Evaluation expression storage unit 130, 230 Attribute evaluation pair candidate storage unit 140, 340 Training case creation 1 unit 150, 350 Model creation 1 unit 160, 260, 360, 460 Classification 1 means 170, 270 Classification 1 means classification result storage unit 180 Training example creation 2 means 190 Model creation 2 means 210 Training example creation means 220 Model creation means 230 Classification means 290 Classification 2 means

Claims (4)

Input a training document including a positive example attribute evaluation pair with a label that associates the attribute expression with the evaluation expression in the document, and candidate attribute evaluation pairs corresponding to the combination of the attribute expression and the evaluation expression existing in the training document For each training document, and for each attribute evaluation pair candidate, the surface character string of the phrase including the attribute expression and the evaluation expression, the part of speech of the phrase including the attribute expression and the evaluation expression, and the attribute expression An attribute evaluation pair candidate extraction means for performing processing to add feature information such as distance information between the attribute expression and the evaluation expression, or whether the clause including the evaluation expression has a dependency relationship;
Enter a training document including a positive example attribute evaluation pair with a label that associates the attribute expression with the evaluation expression in the document, and a candidate attribute evaluation pair grouped for each training document, and the same as the positive example attribute evaluation pair The attribute evaluation pair candidate is determined as a positive example attribute evaluation pair candidate, and the attribute evaluation pair candidate that is not a positive example attribute evaluation pair candidate is defined as a negative example attribute evaluation pair candidate in the group in which the positive example attribute evaluation pair candidate exists. 1 means to create training cases,
Model creation 1 means for inputting a positive example attribute evaluation pair candidate and a negative example attribute evaluation pair candidate and performing a process of creating a classification model 1 such as a discriminant function Z1;
Input a training document including a positive example attribute evaluation pair with a label that associates an attribute expression with an evaluation expression in the document, an attribute evaluation pair candidate grouped for each training document, and a classification model 1 such as a discriminant function Z1 Then, an attribute evaluation pair candidate belonging to a group made up of attribute evaluation pair candidates that does not include any of the same attribute evaluation pair candidates as the positive example attribute evaluation pair is extracted as an undetermined attribute evaluation pair candidate, and the identification function Z1 etc. A classification 1 means for performing a process of classifying an undetermined attribute evaluation pair candidate into a positive example attribute evaluation pair candidate or a negative example attribute evaluation pair candidate using the classification model 1;
Training document including a positive example attribute evaluation pair with a label that associates an attribute expression with an evaluation expression in the document, a candidate attribute evaluation pair grouped for each training document, and a positive example attribute evaluation classified by the classification model 1 means A pair candidate is input, the positive example attribute evaluation pair candidate is replaced with a negative example attribute evaluation pair candidate, and the same attribute evaluation pair candidate as the positive example attribute evaluation pair is determined as a positive example attribute evaluation pair candidate 2 examples of training case creation,
A process of inputting a positive example attribute evaluation pair candidate determined by the training case creation 2 means and a negative example attribute evaluation pair candidate replaced by the training case creation 2 means, and creating and outputting a classification model 2 such as a discriminant function Z2. 2 models to create,
An opinion extraction learning device characterized by comprising: Enter the survey target document that contains the web bulletin board such as blog, E-mail, questionnaire results, inquiry data to the company inquiry window, reports, and other writer's opinions. Attribute evaluation pair candidates corresponding to a combination of existing attribute expressions and evaluation expressions are extracted by grouping for each document to be investigated, and for each attribute evaluation pair candidate, a clause including attribute expressions and evaluation expressions The surface part of the character string, the part of speech of the phrase including the attribute expression and the evaluation expression, whether the phrase including the attribute expression and the phrase including the evaluation expression are in a dependency relationship, or the distance information between the attribute expression and the evaluation expression Attribute evaluation pair candidate extraction means for performing a process of assigning feature information of
Input the attribute evaluation pair candidate grouped for each document to be investigated, and the classification model 1 such as the discriminant function Z1 for determining the boundary plane for dividing the attribute evaluation pair candidate into the positive attribute evaluation pair and the negative attribute evaluation pair. A classification 1 means for performing processing for extracting a positive example attribute evaluation pair candidate from attribute evaluation pair candidates in the same group using the classification model 1;
A positive example attribute evaluation pair candidate extracted by the classification 1 means, and a classification model 2 such as an identification function Z2 for determining a boundary surface for dividing the positive example attribute evaluation pair candidate into a positive example attribute evaluation pair and a negative example attribute evaluation pair A classification 2 means for performing a process of inputting, extracting a positive example attribute evaluation pair from a positive example attribute evaluation pair candidate, and performing a process of determining the positive example attribute evaluation pair as a true attribute evaluation pair;
An opinion extraction classification apparatus characterized by comprising: Input a training document including a positive example attribute evaluation pair with a label that associates the attribute expression with the evaluation expression in the document, and candidate attribute evaluation pairs corresponding to the combination of the attribute expression and the evaluation expression existing in the training document For each training document, and for each attribute evaluation pair candidate, the surface character string of the phrase including the attribute expression and the evaluation expression, the part of speech of the phrase including the attribute expression and the evaluation expression, and the attribute expression An attribute evaluation pair candidate extraction means for performing processing to add feature information such as distance information between the attribute expression and the evaluation expression, or whether the clause including the evaluation expression has a dependency relationship;
Enter a training document including a positive example attribute evaluation pair with a label that associates the attribute expression with the evaluation expression in the document, and a candidate attribute evaluation pair grouped for each training document, and the same as the positive example attribute evaluation pair Attribute evaluation pair candidates in a group including at least one attribute evaluation pair candidate are extracted, and attribute evaluation comparison pair candidates are selected based on the features and positional relationships of the two attribute expressions in the two attribute evaluation pair candidates. When the attribute expression that is generated and exists on the left side from the beginning in the training document matches the attribute expression of the positive attribute evaluation pair in the training document, the attribute evaluation that adds the Left label to the attribute evaluation comparison pair candidate When a comparison pair is generated and the attribute expression existing on the right side from the top in the training document matches the attribute expression of the positive example attribute evaluation pair in the training document, the right evaluation label pair is added to the attribute evaluation comparison pair candidate. A training set created 1 means for performing processing for generating an attribute evaluation comparing pairs of imparting Le,
Model creation 1 means for performing processing for creating a classification model 3 such as a discriminant function Z3 by inputting the attribute evaluation comparison pair labeled with Left or Right by the training case creation 1 means,
Input a training document including a positive example attribute evaluation pair with a label that associates an attribute expression with an evaluation expression in the document, an attribute evaluation pair candidate grouped for each training document, and a classification model 3 such as an identification function Z3 Then, attribute evaluation pair candidates belonging to a group composed of attribute evaluation pair candidates that do not include any of the same attribute evaluation pair candidates as the positive example attribute evaluation pair are extracted as undetermined attribute evaluation pair candidates, and two sets of undetermined An attribute evaluation comparison pair is generated based on the two attribute expression features in the candidate attribute evaluation pair, and using the classification model 3 such as the discriminant function Z3, the generated attribute evaluation comparison pairs are sequentially compared in the tournament method. One attribute evaluation pair candidate finally remaining for each group is set as a positive example attribute evaluation pair candidate, and attribute evaluation pair candidates other than the positive example attribute evaluation pair candidate are set as negative example attribute evaluation pairs. Classification 1 means for performing a process of setting the accessory,
Training document including a positive example attribute evaluation pair with a label that associates an attribute expression with an evaluation expression in the document, a candidate attribute evaluation pair grouped for each training document, and a positive example attribute evaluation classified by the classification model 1 means A pair candidate is input, the positive example attribute evaluation pair candidate is replaced with a negative example attribute evaluation pair candidate, and the same attribute evaluation pair candidate as the positive example attribute evaluation pair is determined as a positive example attribute evaluation pair candidate 2 examples of training case creation,
A process of inputting a positive example attribute evaluation pair candidate determined by the training case creation 2 means and a negative example attribute evaluation pair candidate replaced by the training case creation 2 means, and creating and outputting a classification model 4 such as a discriminant function Z4. 2 models to create,
An opinion extraction learning apparatus characterized by comprising: Enter the survey target document that contains the web bulletin board such as blog, E-mail, questionnaire results, inquiry data to the company inquiry window, reports, and other writer's opinions. Attribute evaluation pair candidates corresponding to a combination of existing attribute expressions and evaluation expressions are extracted by grouping for each document to be investigated, and for each attribute evaluation pair candidate, a clause including attribute expressions and evaluation expressions The surface part of the character string, the part of speech of the phrase including the attribute expression and the evaluation expression, whether the phrase including the attribute expression and the phrase including the evaluation expression are in a dependency relationship, or the distance information between the attribute expression and the evaluation expression Attribute evaluation pair candidate extraction means for performing a process of assigning feature information of
Input the attribute evaluation pair candidate grouped for each document to be investigated, and the classification model 3 such as the discriminant function Z3 that determines the boundary surface for dividing the attribute evaluation pair candidate into the positive attribute evaluation pair and the negative attribute evaluation pair. Then, an attribute evaluation comparison pair is created from two sets of attribute evaluation pair candidates in the same group, and a plurality of generated attribute evaluation comparison pairs are sequentially compared in the tournament method using the classification model 3 such as the discriminant function Z3. One attribute evaluation pair candidate finally remaining for each group is set as a positive example attribute evaluation pair candidate, and attribute evaluation pair candidates other than the positive example attribute evaluation pair candidate are set as negative example attribute evaluation pair candidates Classification 1 means for processing;
A positive example attribute evaluation pair candidate set by the classification 1 means, and a classification model 4 such as an identification function Z4 for determining a boundary surface for dividing the positive example attribute evaluation pair candidate into a positive example attribute evaluation pair and a negative example attribute evaluation pair The classification 2 means for performing the process of inputting and extracting the positive example attribute evaluation pair from the positive example attribute evaluation pair candidates set by the classification 1 means and determining the positive example attribute evaluation pair as a true attribute evaluation pair When,
An opinion extraction classification apparatus characterized by comprising:

Images