JP4677563B2 - Decision support system and decision support method - Google Patents

Description

  The present invention relates to a decision support system that collects and presents useful information related to a human being making a decision on a given theme and supports human decision making.

  In order for human beings to make reasonable decisions on any matter as the environment and systems around the world progress and become more complex, in many cases, more and more evaluations are made based on evaluation criteria from various viewpoints. It is important to make a judgment after collecting and organizing information and considering them comprehensively. For this reason, various decision support systems using computers and databases have been proposed.

  Japanese Patent Laid-Open No. 08-83182 is an alternative in which information relating to an object to be decided is input, the input object information is evaluated by one or more designated software, and the evaluation result or an alternative plan based on the evaluation result is output. A decision support system comprising a plan selection processing means, which learns a case in which a decision is made by the alternative selection processing means, extracts case information including input target information and processing procedure information, and A case storage means for storing and storing the extracted case information is provided, and input target information for each case stored in the case storage means and the input target information in response to input of information related to a new decision-making target Compared with information, select cases with similar target information and output similar case evaluation results as target evaluation results for new decision making. Decision support system which is adapted to disclose.

Japanese Patent Laid-Open No. 2004-185614 receives input information including a plurality of conditions when making a decision as an element, and correlates and correlates each of a plurality of predetermined semantic elements with each element of the input information. By determining the strength and optimizing the correlation and correlation strength of a plurality of predetermined semantic elements according to the target represented by the correlation and correlation strength of each element of the input information, A decision support apparatus is disclosed that is output as decision support information when making a decision.
Japanese Patent Laid-Open No. 08-83182 JP 2004-185614 A

  However, all of these conventional decision support systems extract similar cases in the case from already accumulated case information and a large amount of available information, and organize information related to the case. In addition, it is intended to find the trend or law of correlation strength in the case and the preceding and similar cases and to show the optimized result for the purpose fulfillment based on it. there were.

  For this reason, in the conventional decision support system, it is intended to make a value judgment for each element constituting the case from the related information (same case and similar case) of the case itself that is the subject of decision making. As a result, many people in the world did not widely refer to the thoughts and opinions of the world, and this often resulted in information presentation and proposals that were free from the thoughts and opinions of many people related to the case. It is.

  On the other hand, along with the remarkable spread and development of the Internet in recent years, by browsing the questionnaire results, opinions, reviews, impressions, blogs, etc. posted on the Internet web server for a given theme, many related to the theme Objective information and subjective information can be obtained.

  Text mining is a technique that uncovers the contents of a certain topic from a large amount of information obtained on the Internet and extracts some trends. Text mining is a technique that is mostly related to a given theme. Since objective information and subjective information such as approval or disagreement with respect to the subject were classified and presented, its usefulness was extremely low when it was used as a decision-making tool.

  The present invention makes effective use of a text mining technique and makes judgment information for rational and valid decision support by referring to a plurality of issues based on opinions of many people related to a given theme. It is an object of the present invention to provide a decision support system that is highly useful regardless of the operation method of the operator.

Therefore, the present invention provides a database comprising a web search server having a browser for searching data on multimedia connected to the Internet, a database server for collecting and storing the searched data, and accessing the database. A text mining decision support system configured by a text mining device for an input theme, wherein the text mining device accesses the database and determines whether the input theme is positive or negative. A point extraction means for collecting a group of opinion texts in the database that are pre-classified in the opinion, and extracting a noun phrase and a verb phrase appearing in the group of opinion texts as a plurality of points related to the theme; For multiple issues, include the issue A unique degree calculation means for calculating a specific degree represents the percentage of opinion with one position of positive or negative with respect to the issue among the opinion text group, for each of the plurality of issues, said in the opinion text group Importance calculation for calculating the importance of the opinion with respect to the issue point calculated by calculating the ratio of the appearance frequency of the issue point in the one position among the appearance frequencies of either the positive or negative position Means for extracting content words including nouns, verbs and adjectives appearing redundantly from the opinion text group including the discussion points as related terms related to the discussion points, and the plurality of discussion points A plurality of representative opinions that are affirmative or negative are selected by calculating the frequency of appearance of the content word, and the frequency of appearance of the selected representative opinions is scored. Representative opinion selection means for ranking and outputting the plurality of representative opinions in descending order of the points of the ring, and the plurality of issues and the uniqueness for each issue on the two-dimensional surface of the uniqueness and the importance And an interface unit that outputs image data in which the importance, the related word, and the representative opinion are plotted, and a decision support system based on text mining.

  Here, the database includes a web search server having a browser for searching for data on multimedia connected to the Internet, and a database server for collecting and storing the searched data.

The representative opinion selection means includes means for morphological analysis of all of the opinion text groups including the argument for each position opposite to approval of the argument.

  As described above, in the present invention, the text mining technique is effectively used to collect a group of opinion texts related to the given theme, and a plurality of points related to the theme are selected from the group of opinion texts. By calculating the degree of uniqueness that expresses the proportion of opinions that have a positive or negative position for each issue and the importance of opinions that include the position for each issue, many By making it possible to provide reasonable and valid decision support information by referring to multiple issues based on the opinions of the world, it is highly useful regardless of the operator's operation method. We were able to provide a decision support system.

  This decision support system is particularly effective for themes that include issues related to the latest current affairs, and since the latest opinion texts are collected, the information content collected is not fixed or obsolete. It is possible to provide judgment information for decision support in accordance with the trend of opinions of many people related to.

  Hereinafter, the details of the decision support system and the decision support method according to the present invention will be described.

  FIG. 1 shows a functional block diagram of a decision support system 10 of the present invention. As shown in FIG. 1, the decision support system 10 according to the present invention includes a database including a database server 118, a web search server 19, and a plurality of storage devices (disk devices) 22 to 24.

  As described above, the decision support system 10 uses the browser for searching the data on the multimedia connected to the Internet, the database server 18 for collecting the searched data, and the predetermined data. A database comprising storage means 22 to 24 for storing data is provided, and decision making is supported by performing text mining on a given theme.

  The database server 18 has a beta base management system (DBMS), and under the control of the DBMS, data elements are stored in a predetermined search language (for example, SQL), searched, processed, and output. Note that the firewall 21 shown in FIG. 1 is for preventing unauthorized entry from the outside of the WWW Internet 25.

  The decision support system 10 of the present invention accesses the above-mentioned database, collects opinion text groups related to the theme input by the input means 11, and from among the opinion text groups, a plurality of related to the theme. There is a point extraction means 12 for extracting a point. Here, the argument collection means 12 extracts a plurality of issues related to the theme by extracting noun phrases and verb phrases that appear in the opinion text group related to the input theme. This makes it possible to refer to multiple issues based on the opinions of many people related to a given theme, and as a result, it is possible to provide decision support information that is rational and valid. It is said.

  As a particularly main configuration of the present invention, each of the plurality of issues collected by the issue extraction means 12 has any position such as approval or denial with respect to the issue from a group of opinion texts including the issue. A uniqueness calculating means 13 for calculating the uniqueness representing the proportion of opinions, and an importance calculating means 14 for calculating the importance of an opinion that includes any position such as approval or denial for each of the plurality of issues. It has.

  Here, the uniqueness calculated by the uniqueness calculating means 13 is calculated by obtaining the probability of an opinion having an affirmative position with respect to each of the issues, and the importance by the importance calculating stage 14 is: It is calculated by calculating the ratio of the frequency of appearance of issues in either one of the positive opinions or the negative in all opinion texts. In this way, the tendency and weighting of the opinion for each issue point is made.

  The decision support system further includes a related word extracting unit 15 that extracts related words related to a plurality of issues, and a representative opinion that selects a plurality of representative opinions in affirmative or negative opinions for each of the plurality of issues. Selection means 16 is provided. Here, the related words extracted by the related word extracting means 15 are extracted from the content words such as nouns, verbs, and adjectives appearing in duplicate from the opinion text including the plurality of points discussed above. I have to. Further, the representative opinion selection means 16 performs morphological analysis on all of the opinion text including a certain argument, selects the plurality of representative opinions by calculating the appearance frequency of the content word, and selects the plurality of representative opinions selected. By scoring the appearance frequency, a plurality of representative opinions are ranked and output.

  Then, the interface unit 17 outputs, on the screen of the display device, image data in which the plurality of issues, the related words, or the representative opinion are plotted on the two-dimensional surface of the uniqueness and the importance. For the operator, the decision support information is displayed in a form that is easily visible at a glance. By using this decision support system, the operator can obtain judgment information for rational and valid decision support regarding the input theme regardless of the operation method. It can be done.

  FIG. 2 is a diagram schematically showing the flow from the extraction of subjective information from the collected opinion text group to the output of an information screen supporting decision making in this decision support system. is there. In FIG. 2, the portion surrounded by a broken line is performed in advance by automatic processing. This system inputs opinions on a certain theme. Opinions to be entered shall be classified as either in favor or against the subject theme. Extract “points” from the input opinions, and calculate the degree of specificity and importance for each point. Specificity is a measure that expresses "whether it is discussed in favor or disagreement".

  Importance is a measure of how many people are discussing. In addition, for each of the extracted issues, a related word and a representative opinion are sought. Visualize the distribution of issues by interface.

  FIG. 3 is an example in which the distribution of points of issue is visualized with respect to the theme of “entering hospital management by a corporation”. The issues are displayed on a two-dimensional plane with the importance as the vertical axis and the specificity as the horizontal axis.

  The issues such as “patient” and “medical treatment” near the center of FIG. 3 indicate that both the pros and cons are discussed. The issues such as “competition”, “health insurance”, and “improvement” indicate that the specificities are close to the pros, so that they are the arguments of the pros. Issues such as “profit”, “business”, and “medical corporation” represent issues that are the basis of the opposition because the specificities are close to the opposition. The issues discussed in one position may be unfavorable for the opposite position, or may not be mentioned in one position due to insufficient discussion. By determining the degree of uniqueness, it is possible to find out what is the reasoning for each position, and at the same time to discover issues that are likely to be insufficiently discussed. Also, looking at the points of opposition in FIG. 3, “medicine” has the highest importance, followed by “profit”, “business”, and “profit”. This means that many opposition people think these issues are important. By finding the importance, you can discover what many people think important and discuss.

  When an issue point visualized on the interface based on uniqueness and importance is selected, related words and representative opinions of the issue point are displayed. FIG. 4 shows a display example of a related word of the issue of “information” in the theme of “entry into hospital management of a corporation,” and FIG. 5 shows a display example of a representative opinion. In FIG. 4, related words such as “for-profit” and “medical” are displayed around “information”. Related issues such as “medical” and “business” are also displayed. In FIG. 5, representative opinions discussed about “information” are selected and displayed from the pros and ops. Therefore, the user can know how the issue of “information” is actually discussed.

  On a bulletin board on the Web or the like, a user who has participated in the discussion from the middle may make a statement about the points already discussed. On the other hand, responses such as “The point has already been considered” or “See the post on the day of the month” may be returned. If the points discussed in the past can be grasped and representative opinions on important points can be browsed, such a wasteful exchange as described above can be avoided.

  In the following, A to E explain each of the processing of “extraction of points”, “calculation of uniqueness”, “calculation of importance”, “extraction of related words”, and “selection of representative opinions”, and F Describes the interface. G explains the application to multi-stage evaluation.

A. Extraction of issues (1) Outline Prior to the implementation of the system, in order to confirm the format in which issues are expressed, we manually analyzed the opinions posted on women and BS debates. As a result, it was found that the range of issues is various such as “word”, “phrase”, “dependency (sentence)”, “sentence”, and the like. The complexity of the model of issues handled in this system increases the expression ability. However, the analysis accuracy is lowered. Therefore, the issues expressed by “word”, “phrase”, and “dependency” that can be extracted with a small amount of errors by the current natural language processing technology are extracted. Specifically, noun phrases and verb phrases in the opinion are extracted. FIG. 6 shows an example of issues extracted when one of the objections to the theme of “entry into hospital management of a corporation” is actually input to this system. Here, verb phrases are unified in the form of “noun_verb” by deleting particles. Details will be described in (3).

(2) Extraction of noun phrases In order to extract noun phrases, opinion text is morphologically analyzed by ChaSen (http://chasen.naist.jp/hiki/ChaSen), and part-of-speech information is used. ChaSen is a tool that morphologically analyzes an input sentence and outputs morpheme, reading, original form, and part of speech information. FIG. 7 shows an output example of ChaSen when “It is important to disclose information” is input. “Noun-general”, “noun-sa-variant connection”, and “noun-adjective verb stem” were added as parts of speech for “information”, “public”, and “important”, respectively. In this way, even finer classifications are output for nouns. Using these parts of speech information, noun phrases are constructed and extracted as points of argument.

  Specifically, if the patterns listed in Table 1 appear in the opinion text, they are collected as noun phrases. “/” Represents a morpheme boundary. (A) to (f) in Table 1 are patterns that are simply connected, and (g) is a pattern that performs paraphrasing. (G) is a rule for unifying expressions having the same meaning even if the notation is different.

  By omitting the particles between the nouns and the nouns of the sub-variable stems, the expressions “information disclosure”, “information disclosure”, and “information disclosure” are all unified to “information disclosure”. When patterns that match any one of (a) to (g) are continuous, each pattern is extracted individually. For example, “medical expenses”, “high medical expenses”, and “reduced medical expenses” are exhaustively extracted from the character string “reduced high medical expenses” by (a), (e), and (g). This is because it is difficult to determine the range of noun phrases including adjectives, adjective verbs, and sa-variants as points to be extracted.

  When the noun phrase composed of the pattern of (a) is composed of three or more nouns, only the longest noun phrase is extracted. For example, in the case of “monopoly / prohibition / law”, “monopoly prohibition” and “prohibition law” are not extracted, but only “monopoly prohibition law” is extracted. This is because a person who discusses the “antitrust law” always uses the expression “antimonopoly law” and does not use the expression “antimonopoly” or “prohibition law”. Noun phrases included in the theme tend to appear frequently in opinion texts even if they are not issues. Therefore, noun phrases included in the theme are not extracted as points of argument.

  In addition to a noun phrase composed of a plurality of words, a single noun is also treated as an argument. However, in order to exclude common words, only nouns with two or more letters are targeted, and nouns that appear only once in the position of approval or disagreement are not extracted as points of argument.

  In addition, nouns analyzed by ChaSen as “numerical”, “pronoun”, “suffix”, “non-independent”, “adverb possible”, “Nay adjective stem” are deleted.

  Furthermore, words that are “not an issue” in all themes were collected empirically and used as stop words. Stopwords are not an issue alone. However, when constructing a noun phrase together with other words, it may be extracted as an issue point. Table 2 shows the stop words.

(3) Extracting verb phrases Regarding the theme of “entering hospital management by a corporation”, if you just look at the noun “profit”, you do n’t know whether you want “profit” or “ignore profit”. It may not be appropriate as an issue. Therefore, dependency analysis of opinion text is performed using CaboCha (http://chasen.org/~taku/software/cabocha/), and a verb phrase is extracted based on the analysis result.

  FIG. 8 shows an example in which a sentence “Medical care is not suitable for profit pursuit” is analyzed by CaboCha. The number after “*” represents the number of the clause, and the next number represents the number of the clause related to the clause. Each clause is morphologically analyzed and the part of speech information of each morpheme is output. In the case of “medicine is not suitable for profit pursuit”, “medical is” and “for pursuit of profit” are related to “not suitable for pursuit of profit”. Using these pieces of information, we extract the expressions in which the clauses composed of nouns and particles are related to the verbs.

  However, if the noun that is the subject or object is “pronoun” or “non-independent”, it is not extracted. Also, if the particle and verb are “to +”, or if the extracted verb phrase is a high-frequency expression such as “problem” or “worry”, it is not extracted as an argument.

  In the extracted verb phrase, even if the particles are different, such as “pay money” and “pay money”, the same expression is unified, so the particle is deleted and replaced with “_”.

  In the method (2), the sa-verb is treated as a noun phrase, so the verbs other than the sa-verb are treated as verb phrases. For example, from the sentence “eliminate waste and reduce costs”, verb phrases “eliminate waste” and “reduce costs” are extracted. However, “reducing expenses” is extracted as a noun phrase “reducing expenses” by the method (2). Therefore, from this sentence, only the argument “no use_exclusion” in which the particle is omitted from “no use” is extracted.

B. Calculation of the Specificity The uniqueness of the issue A is a measure that represents “from which standpoint the issue A is discussed.” The degree of peculiarity of issue A is defined as the conditional probability that “when one opinion is selected, the opinion discusses issue A and is the opinion of the proponent” and “when one opinion is selected, Calculation is based on the difference in conditional probability that the opinion is about point A and is the opinion of the opponent. Specifically, the uniqueness of the issue point A is calculated by Equation 1.

数１中の条件付き確率は、数２を用いて計算する。

The conditional probability in Equation 1 is calculated using Equation 2.

ここで、数２中のXは賛成（ｐｒｏ）か反対（ｃｏｎ）のどちらかである。数１は確率の差であるから、−１以上１以下の値をとる。そのため、賛成派だけが論じている論点の固有度は１、反対派だけが論じている論点の固有度は−１となる。両方の立場で均等に論じられている論点の固有度は０に近い値をとる。

Here, X in Formula 2 is either pro (pro) or opposite (con). Since Equation 1 is a difference in probability, it takes a value between −1 and 1. Therefore, the uniqueness of the issue discussed only by the pros is 1, and the uniqueness of the issue discussed only by the opponents is -1. The peculiarities of issues that are equally discussed in both positions are close to zero.

  Table 3 shows examples of issues and specificities extracted from the theme of “participation in hospital management of a corporation”. “Health insurance” and “customer needs” with a uniqueness close to 1 are mainly arguments for the pros, while “profit pursuit” and “medical corporation” with a uniqueness close to -1 are mainly for the opposition. It is. Since the degree of uniqueness of “Cosmetology” is 0, it is a neutral issue discussed by both the pros and cons.

C. Calculation of importance The importance of an issue has different values for the pros and ops. The importance of the issue point A in the position X is a scale that represents “how many people are discussing the issue point A in the position X”.

I think that issues with more people are the more important issues in that position.
Specifically, the calculation is performed using Equation 3.

  However, first, the uniqueness of the issue A is calculated using Equation 1, and the issue A is fixed in a position where the uniqueness is large. Then calculate the importance of that position. However, for issues with a uniqueness of 0, the importance is obtained for each position in favor and opposite, and these are averaged to obtain the final importance.

Within one theme, issues that are much more important than other issues are often common nouns that often appear in the theme. Therefore, empirically, nouns with an importance of 20.0 × 10 ⁻³ or more are not extracted as points of argument.

  Table 4 shows examples of issues and importance extracted from the theme of “participation in hospital management of a corporation”. The four cases from “improvement” to “customer needs” are the issues of the pros and cons, and the four cases from “medical corporation” to “profit pursuit” are the issues of the opposition. Since the degree of uniqueness of “Cosmetology” is 0, the average of the importance calculated from the standpoint of approval and the opposite is used as the importance. It can be seen that the pros discuss mainly “improvement” and the dissent mainly discuss “medical corporation”.

D. Extraction of related words “Related words” may be useful when the meaning of a single word is unknown or when various associations are made from one word. Therefore, a function to extract and display related words of each issue point from the opinion is provided. In the present decision support system, two of “related words” and “related issues” are extracted as related words. In the case of issues discussed by both the pros and cons, the related terms are extracted after fixing their positions based on their uniqueness.

(1) Extraction of related content words In the theme of “entry into hospital management of a corporation”, “improvement” is extracted as an issue.
The subject of “improvement” in this theme is the current status of “medicine” and “doctor”. Therefore, the issue of “improvement” and “medicine” and “doctor” are highly related and often co-occur in the same opinion. Therefore, the content words that frequently appear in the opinion in which the point A is discussed are extracted as the related words of the point A. Content words are nouns, verbs, and adjectives in this decision support system.

  However, in order to exclude common words, the nouns listed in Table 2 and frequently used verbs such as “do” and “naru” are manually defined as stop words and excluded from related words.

  Simply counting the frequency at which the issue A and the content word B appear in the same opinion does not give an accurate relevance. This is because A and B are not always related when only one of the appearance frequencies is high. Therefore, it is necessary to consider the co-occurrence frequency of A and B. In this method, the degree of association between the issue point A and the content word B is calculated by the dice coefficient shown in Equation 4.

  F (A) is the number of opinions in position X where issue A appears, F (B) is the number of opinions in position X where content word B appears, and F (A, B) is the co-occurrence of issue A and content word B Indicates the number of opinions of position X. The greater the value of Equation 4, the higher the degree of association between A and B.

  Table 5 shows examples of issues and related words extracted from the theme of “entry into hospital management of a corporation”.

(2) Extraction of related issues In the case of the theme of “entering hospital management of a corporation”, the issue of “customer needs” and the issue of “cosmetic surgery” It is often discussed together in the same opinion. Extract a set of points that are often discussed together in the same opinion.
The degree of relevance between issue point A and issue point B is calculated using the Dice coefficient, as in Equation 4.

  Table 6 shows an example of issues related to issues extracted from the theme of “entering hospital management of a corporation”. For one issue, there are words that are extracted as both “related words” and “related issues”.

E. Choosing a representative opinion When a user wants to know in what context an issue is actually discussed, it is inefficient to read all the opinions that contain that issue. Therefore, a representative opinion is selected from the opinions including the issue point designated by the user.

  In this decision-making support system, the opinion text group including the specified issue points reflects the tendency of the opinion group as a whole and is considered to be more representative.

The process of selecting a representative opinion from the opinions including the issue A is performed according to the following procedure.
1. In each of the opposite positions, the opinion text including issue A is morphologically analyzed to determine the frequency of occurrence of content words.

2. The score of opinion X is calculated using Equation 5. The higher this score, the more important the opinion.

長い意見ほど内容語を多く含んでいるので、有利になってしまう。そこで、その意見が含んでいる形態素の数によって正規化することで、意見の長さによる差を吸収する。

Longer opinions are more advantageous because they contain more content words. Therefore, normalization is performed by the number of morphemes contained in the opinion to absorb the difference due to the length of the opinion.

3. Sort the opinions in descending order of score and output as a ranked list.
This processing is performed in a position opposite to and in favor, and “representative opinion in favor” and “representative opinion in the opposite side” are displayed. The user can then compare the opinions of the pros and cons on the same issue.

  In Expression 5, there is a possibility that the accuracy of selecting a representative opinion can be increased by reducing the weight of the content word included in the opposite position. However, further consideration is for further study.

F. Using the methods described in Interfaces A to E, “the issues”, “the importance of each issue”, “the uniqueness of each issue”, “related words”, and “representative opinions” are obtained from the input opinion text group. It is done. An interface that aggregates and displays these is implemented by Flash.

  FIG. 3 is a diagram showing the points of concern regarding the theme of “entering hospital management of a corporation” with the uniqueness as the horizontal axis and the importance as the vertical axis. The issues displayed here are issues extracted from the input opinion that are higher than the threshold that determines the importance in each position.

  The number of issues to be displayed can be adjusted by designating the lower limit of the importance of issues from the operation panel on the right side.

  FIG. 4 shows the result of displaying the related points of the related points and the related points. Clicking on an issue point displays the top four words with the highest degree of relevance. In FIG. 4, “profit”, “medical”, “company”, and “public” are displayed as related words of “information”. Related words can be displayed and compared for multiple issues at the same time. The right operation panel displays the uniqueness and importance values of the clicked issue. When “display related issues” is selected from the operation panel, the color of the issues with high relevance displayed on the spot changes. In FIG. 4, the issues “information disclosure”, “medical care”, “profit”, and “company” are displayed as issues related to “information”.

  FIG. 5 shows the result of displaying a representative opinion from among the opinions discussing the designated issue point. Click on the issue you wish to examine in detail, and select “View representative opinions” from the operation panel. The top 3 items in the ranked list obtained by the E method will be displayed from the opinions discussed on the issue. The If the specified issue is discussed in both favor and disagreement, a representative opinion in the position of approval and a representative opinion in the opposite position are displayed simultaneously as shown in FIG. The user can compare how the specified issues are actually discussed in their respective positions.

G. Application to multi-level evaluation By classifying reviews of multi-level evaluation such as “5 stars” into pros and ops according to the number of stars, it is possible to input to this system. Figure 9 shows reviews of the movie Star Wars Episode 3 (http://moviessearch.yahoo.co.jp/detail?ty=mv\&id=321602) with 1 to 3 stars Is “opposite”, and 4 to 5 reviews are regarded as “agree” and are input. The names of characters such as “Anakin” and “Obiwan”, “battle scene”, “story development”, etc. were extracted as issues.

H. Evaluation Experiment (1) Overview There are two system evaluation methods called “black box test” and “glass box test”. The black box test is a method for evaluating the performance of a system only with input and output. However, when the system is composed of a plurality of elements, it is necessary to evaluate the contribution of each element individually. Therefore, the glass box test individually evaluates the performance of each element constituting the system. When evaluating the system, it is necessary to perform these two evaluations.

  The system created by this decision support system is intended for human decision support. For this reason, when performing a black box test on this system, it is necessary to have the subject use the system to evaluate how useful it was for decision making. However, the influence of decision making on the user differs depending on whether the subject is an individual or a group. Also, the results obtained may differ depending on the theme. That is, there are many “indeterminate elements”. Evaluation work such as TREC (http://trec.nist.gov/) and NTCIR (http://research.nii.ac.jp/index-en.html) are examples of conducting evaluation experiments with many uncertainties. There is a shop. These workshops will develop evaluation data and methods for information retrieval and natural language processing over a period of one to one and a half years. In other words, establishing a method for an evaluation experiment itself is one research theme. Therefore, this decision support system only performed the glass box test, not the black box test. Specifically, “extraction of points”, “paraphrase of noun phrases”, “calculation of uniqueness”, and “selection of representative opinions” were evaluated.

  Opinion text on current affairs was used for the evaluation data. All of them were collected manually from a website that published opinion information posted by an unspecified number of users. Table 7 shows the themes used, the number of opinions, and the sources. For “Do you agree with the privatization of the postal service”, the evaluation was made by combining the 30 opinions posted on the ewoman and the 18 opinions posted on the Prime Minister's official website. In the following description, for the sake of simplicity, each theme is distinguished by the keyword shown in “abbreviation” in Table 7. In order to increase the objectivity of evaluation, we asked two judges to evaluate each theme.

(2) Evaluation of “extracting issues” a) Experimental method This decision support system extracts noun phrases and verb phrases in opinions as issues, and expresses the distribution of issues on a two-dimensional plane. Visualize the composition with which. Therefore, the extraction of the issue point has a great influence on the performance of the entire system. Specifically, the following viewpoints are evaluated.

i) Is the method of extracting the issues as noun phrases and verb phrases appropriate? ii) How much the system can extract the issues extracted by humans The purpose of “extract issues” is to extract the issues extracted by humans into the system To extract accurately and exhaustively. Therefore, accuracy and completeness are calculated by “accuracy” and “reproducibility”, respectively.

  For each theme, the issues extracted by the system and the issues extracted by the determiner are compared, and the accuracy and recall are calculated using Equations (6) and (7).

  The system first categorizes each extracted issue into a position with higher specificity (in favor or opposite). Then, a plurality of argument groups classified into the pros and cons positions are sorted based on the importance to create a ranked list. Finally, the top N issues are extracted for each list against and against, and the accuracy and recall are calculated. Note that changes in accuracy and recall are observed while increasing N in stages. Judges were given a list of pros and cons for the themes shown in Table 7.

  The judges read each opinion and extracted the points that they thought were the points of discussion. The point of discussion was defined as the “part of which is the argument for or against” each opinion. The unit to be extracted is not limited to a word, a phrase, or a sentence. Also, if there were multiple points in one opinion that seemed to be points of argument, they were all listed.

  FIG. 10 shows an example of issues extracted by the determiner. FIG. 10 is one of the objections to the theme of “entering hospital management of a corporation”, and the underlined part is a point extracted by the judge.

b) Experimental results Table 8 shows a breakdown of the number of points extracted by the judges and the system. The number in parentheses in the “judgment” column represents the number of points that completely match the points extracted by the system.

これらの数字から、判定者ごとに精度と再現率を求め、判定者を横断して平均した値をそのテーマでの精度と再現率とした。表９に各テーマの精度とその平均を、表１０に各テーマの再現率とその平均を示す。さらに、テーマを横断して、精度と再現率をそれぞれ平均して描いた再現率・精度グラフを図１１に示す。

From these numbers, accuracy and recall were obtained for each judge, and the average value across the judges was taken as the accuracy and recall for that theme. Table 9 shows the accuracy and average of each theme, and Table 10 shows the recall and average of each theme. Further, FIG. 11 shows a reproduction rate / accuracy graph drawn by averaging the accuracy and the reproduction rate across the themes.

重要度が上位の論点ほど精度が高く、下位の論点ほど精度が低くなる傾向が出た。また、再現率は重要度が上位であるほど低く、下位にいくほど高くなった。このことから、ユーザがインタフェース上に表示される重要度の下限を調節することにより、精度と再現率のどちらかを優先して表示することが可能であることがわかった。

There was a tendency for the issues with higher importance to be more accurate and for issues with lower importance to become less accurate. In addition, the recall was lower as the importance was higher and higher as the importance was lower. From this, it was found that either the accuracy or the recall can be displayed with priority by adjusting the lower limit of the importance displayed on the interface by the user.

  As a comparison object of this system, the accuracy and recall of the judge were calculated. Specifically, the issue point extracted by one judge was regarded as the correct answer, and how accurately and comprehensively the other judge could extract it was evaluated. Table 11 shows the breakdown by theme. The average of all themes was 0.10 for both accuracy and recall. The accuracy and recall of this system were 0.50 and 0.48, respectively.

図１２〜１３に、判定者間で抽出する論点が異なった例を示す。図１２と図１３はともに、「株式会社による病院経営への参入」というテーマに対する賛成意見の１つである。図１２の下線部は判定者Ａ、図１３の下線部は判定者Ｂが抽出した論点を表している。判定者Ａ、判定者Ｂともに、「情報公開」と「モラルの低さ」を論点として抽出した。しかし、「医療ミス」と「信用」は判定者Ａだけが抽出し、「高いモラル」は判定者Ｂだけが抽出した。

12 to 13 show examples in which the points to be extracted are different among the judges. Both FIG. 12 and FIG. 13 are one of the opinions in favor of the theme of “entry into hospital management by a corporation”. The underlined portion in FIG. 12 represents the issuer A, and the underlined portion in FIG. For both the judges A and B, “information disclosure” and “low morality” were extracted as issues. However, only the judge A extracted “medical error” and “trust”, and only the judge B extracted “high moral”.

  The number of exact matches has decreased because the units to be extracted as issues vary depending on the judge. For example, from the opinion that “severe working conditions are improved”, both judges have extracted issues centered on the word “working conditions”, but the extracted points are “severe working conditions” and “ The notation is different, for example, “Improved conditions”. On the other hand, the system comprehensively extracts different points such as “harsh working condition”, “harsh working condition improvement”, “working condition”, and “working condition improvement”. As a result, the accuracy and recall of the system were higher than the judges.

  Of the issues extracted by humans, 78.5% were noun phrases, 2% were verb phrases, and 19% were other. From this result, it can be seen that the present method, which handles noun phrases and verb phrases as issues, covers about 80% of the issues extracted by humans at best. Therefore, it was found that this method, which treats noun phrases and verb phrases as arguments, is effective. The percentage of verb phrases was as low as 2%. However, since it is included in the issues extracted by humans and there are few absolute numbers in the opinion, it is considered that there is no harmful effect even if it is extracted. In addition, the usefulness of verb phrases is qualitatively clear because there are issues that are meaningless unless extracted as verb phrases, such as “it takes time”.

c) Error analysis The percentage of issues that could be extracted by the system among the issues extracted by humans was 60% for noun phrases and 44.3% for verb phrases. Among the issues extracted by humans, the issues that could not be extracted by the system even though they were noun phrases or verb phrases and the breakdown of issues that could not be extracted because they were not noun phrases or verb phrases were expressed as `` One side of the escalator. The theme “walking” is shown in Tables 12-14.

表１２〜１４に挙げた例は、いずれも、判定者は抽出したものの、システムは抽出できなかった論点である。（ａ）〜（ｊ）は、いずれも抽出対象にしていなかったことに起因する誤りである。そのため、これらのパターンを抽出規則に加えることで抽出は可能になる。しかし、副作用として、正解の数よりもノイズが増える可能性がある。特に、（ａ）、（ｅ）、（ｈ）に対応すると、論点ではない一般的な語が大量に抽出されるため、ストップワードの整備が重要となる。

The examples listed in Tables 12 to 14 are all issues that were extracted by the judge but could not be extracted by the system. (A) to (j) are errors due to none of the extraction targets. Therefore, extraction becomes possible by adding these patterns to the extraction rule. However, as a side effect, noise may increase more than the number of correct answers. In particular, when (a), (e), and (h) are dealt with, a large number of general words that are not points of argument are extracted, so that the maintenance of stop words is important.

(3) Evaluation of “paraphrasing of noun phrases” a) Experimental method The system created by this decision support system unifies the points that have the same meaning even if they have different notations when extracting the points. Paraphrases a noun phrase that includes a stem noun as follows:
Noun + (ga | o | ha | no) + sa variation verb stem → noun + sa variation verb stem By this rule, for example, “information disclosure”, “information disclosure”, “information disclosure” are all “information disclosure” To be unified.

  However, not all noun phrases subject to the paraphrase rule can be paraphrased by this rule. For example, in the noun phrase “invasion of other countries”, “invasion” is analyzed as a noun of a sacramental word stem, which is called “invasion of other countries” in this system. However, “invasion of other countries” before paraphrase means “invasion by other countries”, whereas “invasion of other countries” after paraphrase can also mean “invasion to other countries”. Therefore, it is necessary to evaluate how appropriate the paraphrasing by this rule is.

  Of all the issues extracted by the judge from the opinions on each theme in Table 7, 58 issues that were reworded by this system were evaluated. Each judge point and the opinion of the source of the extraction were both viewed by the judge, and he was asked to determine whether the paraphrasing was appropriate in the context of the opinion as the source of the extraction in the following three stages.

i) A: The meaning does not change, there is no sense of incongruity ii) B: There is a sense of incongruity but is acceptable iii) C: The meaning changes, there is a sense of incongruity Regardless of which issuer the issuer extracted, We asked all judges to determine all the issues to be paraphrased. In addition, when the same issues were extracted from different opinions, or when the same issues were extracted from different themes, they were judged according to the context of the opinions from which they were extracted.

b) Experimental results Table 15 shows the correct answer rate of the present method when the average of the determination results of the two judges is regarded as the correct answer. The accuracy rate of the entire evaluation target was 46.6% when only A was considered correct, and 74.2% when A and B were considered correct.

判定された名詞句の内訳を表１６〜１８に示す。括弧内の数字は、その論点が複数の意見から抽出された場合の意見数を表す。今回の評価対象には、名詞間の助詞が「は」であった例は存在しなかった。

Tables 16 to 18 show the breakdown of the determined noun phrases. The numbers in parentheses represent the number of opinions when the issue is extracted from multiple opinions. In this evaluation, there was no example where the particle between nouns was “ha”.

異なる意見から同じ論点が抽出されている場合や、異なるテーマにおいて同じ論点が抽出されている場合は、抽出元である意見ごとに判定を行ってもらった。しかし、今回の評価実験では、同じ論点を言い換えた結果の適否が意見によって異なる例はなかった。

When the same issue point was extracted from different opinions, or when the same issue point was extracted from different themes, we asked each opinion from which it was extracted. However, in this evaluation experiment, there were no examples in which the appropriateness of the result of rephrasing the same issue differs depending on the opinion.

  From the results of this experiment, it was not possible to find any kind of law based on the sequence determined as “A” and “C” and the difference between particles.

  The current system paraphrases everything for which paraphrasing rules can be applied. However, there is no need to rephrase in the first place if there is no “same notation but same meaning” issue in the same theme. By applying a rule only when paraphrasing is necessary, errors due to paraphrasing can be reduced. It is also effective to check the appearance frequency of the noun phrase before paraphrasing and the noun phrase after paraphrase with a corpus and unify them into expressions with high appearance frequency.

(4) Evaluation of “Calculation of Specificity” a) Experimental Method In this system, the specificity of the issue point A is calculated based on whether the probability of the issue point A appears to be higher or not. The peculiarity takes a value between −1 and 1 inclusive. If it is positive, it indicates an issue that is specific to the pros and negative if it is negative.

  However, even if argument A is discussed in the opinion in favor, it may actually be described as an argument of the opposite position, such as “I agree, but not A”. In such cases, the uniqueness calculated by this method may not match the “true position” to which the issue belongs.

  Also, some opinions that include issue A include only A but do not actually discuss A. In this method, since the uniqueness is calculated for “an opinion including A”, the ideal uniqueness for “an opinion discussing A” may be different from the value.

Therefore, we evaluated how appropriate the uniqueness calculated by this method is.
The uniqueness is a continuous value and represents “agreement or opposite degree”. However, it is difficult for humans to determine the “degree of approval or disagreement” for each opinion. Therefore, as an indirect evaluation method, the position to which the issue point A determined by humans belongs is compared with the position to which the issue point A determined by the uniqueness of this method belongs.

  When the judge determines that the point of issue is extracted from each opinion, if the point of view is considered to be the reasoning of the opposite position to the opinion from which the source opinion belongs, clearly indicate that fact. received. For example, in the case of an opinion of “I agree, but there is a concern that I will continue to pursue profits” that belongs to the group of the theme “Entering into hospital management by a corporation”, “Profit pursuit” is an argument of the opposition , "In pursuit of profit (reverse)" was extracted. In this case, the true position for “profit pursuit” is “opposite”. We compared the “true position” by the judges with the position determined by the system.

b) Experimental results Table 19 shows the judgment results for each judge. “The number of points extracted by the system” represents the number of points that are completely matched with the points extracted by the system among the points extracted by the judge. “Number of issues with the same position” represents the number of issues where the “true position” by the judge matches the position determined by the system. “Accuracy rate” represents the accuracy rate of classification according to the degree of uniqueness.

テーマごとに、判定者の判定による「真の立場」とシステムが決定した立場が一致した論点数の割合を平均し、それらをさらにテーマを横断して平均し、システムの正解率を求めた。全テーマを総合した正解率は９５．６％であった。

For each theme, we averaged the percentage of the number of issues that matched the “true position” determined by the judge and the position determined by the system, and averaged them across themes to obtain the correct answer rate of the system. The correct answer rate for all themes was 95.6%.

(5) Evaluation of “Selection of Representative Opinion” a) Experimental Method The system proposed in this decision support system is a representative opinion out of opinions discussing the point selected by the user on the interface. It has a function to display. In order to achieve the objective of “Understanding all the opinions without having to read all the opinions”, it is necessary to prioritize and display more important opinions. Therefore, the opinion judged as “representative” by humans is compared with the opinion judged “representative” by this method, and whether or not the representative opinion selected by this method is really representative is evaluated.

  Similar to the evaluation of “extracting issues”, the judges were given the themes listed in Table 7 and a list of their opinions in favor and disagreements. And I asked all the opinions that I think are discussing about the issue A about the issue A extracted from the opinions. At this time, if there are multiple opinions discussing issue A, the most important opinion was selected as the “representative opinion”. When there were multiple opinions on the issue A, both in favor and against, we asked them to select the opinion that they thought was important from their respective positions. Also, if there were multiple opinions that I thought were important, I asked them to give them all.

  If there are issues that have the same meaning even though the notation is different, the representative opinion was chosen from all the opinions that discuss each of those issues. For example, if it is determined that “information disclosure” and “information disclosure” have the same meaning, if there are three opinions discussing “information disclosure” and two opinions discussing “information disclosure” The representative opinion of the group of issues consisting of “information disclosure” and “information disclosure” was selected from the above five cases. This is to prevent a case in which, when a representative opinion is considered for each notation, there is only one opinion that discusses the issue, and there is no comparison object that is considered important. For this reason, the input to the system for the evaluation experiment also includes the five opinions discussed on the “information disclosure” and “information disclosure” examples, and simply “information disclosure” or “information disclosure”. An input that contains Then, we investigated whether the opinion that humans judged to be the most representative was judged to be representative in the system.

Each opinion was classified as follows according to the evaluation of the judges.
i) A: A representative opinion among the opinions discussed about the issue ii) B: An opinion that is discussed but not representative iii) C: Even if the issue is included, the issue Undiscussed Opinions Judges choose representative opinions from among those who have judged that they are “discussing”. However, the system ranks the opinions that “include” the issue in importance, and therefore ranks opinions that only contain and are not discussed (corresponding to C). Therefore, for the top 10 items in the ranked list output by the system, the accuracy and recall for the opinions judged by the judge as “discussing” and the opinions judged as “representative” were examined in order from the top. . There are two types of correct answers: “A only” and “A and B”.

  The accuracy and recall when calculating up to the N-th rank in the ranked list as the output of the system are calculated by equations 8 and 9.

ただし、論じている意見が複数ある論点と意見の対だけを対象とし、論じている意見が１つしかない論点と意見の対は対象としなかった。

However, only a discussion point and opinion pair with multiple opinions being discussed were included, and a discussion point and opinion pair having only one argument were not included.

b) Experimental results Tables 20 to 23 show the accuracy and recall obtained by calculating the average among the judges for each theme and averaging the results across the themes. FIG. 14 shows a graph of accuracy and recall that averages all themes.

また、ランダムに意見を並べた場合の精度と本システムで順位付けした場合の精度を表２４に示す。表２４は、上位３件の意見のうち、各順位までの意見を出力とみなした場合の精度である。

Table 24 shows the accuracy when opinions are arranged randomly and the accuracy when ranking is performed by this system. Table 24 shows the accuracy when opinions up to each rank among the top three opinions are regarded as output.

図１４より、Ａだけを正解とした場合と、ＡとＢの両方を正解とした場合によらず、上位の意見ほど高い精度が得られた。また、表２４より、ランダムに意見を並べた場合よりも、本システムによって順位付けした場合の方が高い精度が得られた。しかし、再現率は、Ａだけを正解とした場合と、ＡとＢの両方を正解とした場合でほぼ同じ曲線となった。すなわち、論点Ｘを含んでいる意見内におけるＡの分布と、Ｂの分布はほぼ同じであることがわかった。Ａがより上位に分布していることが理想であるため、本手法にはまだ改善の余地がある。

As shown in FIG. 14, the higher the opinion, the higher the accuracy, regardless of whether only A is correct or both A and B are correct. Further, from Table 24, higher accuracy was obtained when ranking was performed by the present system than when opinions were randomly arranged. However, the recall was almost the same curve when only A was correct and when both A and B were correct. That is, it was found that the distribution of A and the distribution of B in the opinion including the issue point X are almost the same. Since it is ideal that A is distributed higher, this method still has room for improvement.

  Table 25 shows, for each theme, the average number of opinions including each issue point, and the average ranking of opinions determined as A. The number of opinions including each issue point is a value obtained by averaging the number of opinions including the issue point group extracted from each theme by the determiner. For example, in the case of “performance-based”, the average number of opinions including the issue X is 6.82, and the average of the opinions that humans have determined as A is 3.04. .

表２５において、「含んでいる意見数」の括弧内にある数は、含んでいる意見数の中央値を表している。この数字は、本手法による意見の重要さの順位付けを行わず、無作為に出力した結果の平均である。全テーマに対するＡの意見の平均順位は３．２７であり、無作為の順位である２．７２を下回ってしまった。しかし、テーマごとに見ると、５件中３件のテーマ（成果主義、病院経営、郵政民営化）ではＡの平均順位が高くなった。このことから、本手法が有効に働く場合とそうではない場合があることがわかった。今後は、評価対象のテーマ数を増やして、さらなる分析を行う必要がある。

In Table 25, the number in parentheses of “the number of opinions included” represents the median of the number of opinions included. This number is the average of the results that were randomly output without ranking the importance of opinions by this method. The average ranking of A's opinions on all themes was 3.27, which was below the random ranking of 2.72. However, looking at each theme, the average ranking of A was higher in 3 out of 5 themes (performance-based, hospital management, postal privatization). From this, it was found that this method may or may not work effectively. In the future, it will be necessary to increase the number of themes to be evaluated for further analysis.

  As described above in detail, the decision support system according to the present invention effectively utilizes a text mining technique, collects opinion text groups related to a given theme that has been input, and from among the opinion text groups, By extracting multiple issues related to the theme and calculating the uniqueness that represents the proportion of opinions that have affirmative or negative status for each issue and the importance of opinions that include the status for each issue This makes it possible to provide rational and valid decision support information by referring to a plurality of issues based on opinions of many people related to the subject.

  This decision support system is particularly effective for themes that include issues related to the latest current affairs, and since the latest opinion texts are collected, the information content collected is not fixed or obsolete. It is possible to provide decision information for decision support in accordance with the trend of opinions of many people related to.

  The present invention relates to a decision support system using a computer and a database that collects and presents useful information related to human decision making on a given theme and supports human decision making. And has industrial applicability.

1 shows a configuration functional block diagram of a decision support system 10 of the present invention. In this decision support system, the flow from extracting subjective information from collected opinion texts to outputting an information screen supporting decision making is schematically represented. A display example of the distribution of argument points is shown. A display example of related terms is shown. A display example of a representative opinion is shown. An example of extracted issues is shown. An example of morphological analysis is shown. An example of dependency analysis is shown. The execution example which input the review of multistage evaluation is shown. An example of the issues extracted by the judge is shown. The example of the recall-precision graph of argument extraction is shown. The example of the issue point which the judge A extracted is shown. The example of the issue point which the judge B extracted is shown. Examples of accuracy and recall for all themes are shown.

Explanation of symbols

10: this decision support system 11: input means 12: argument extraction means 13: fixed degree calculation means 14: importance calculation means 15: related word extraction means 16: representative opinion selection means 17: interface means 18: database server 19: Web search server 20: gateway server 21: firewall 22: storage device (1)
23: Storage device (2)
24: Storage device (3)
25: WWW Internet

Claims (4)

A database comprising a web search server having a browser for searching data on multimedia connected to the Internet, and a database server for collecting and storing the searched data, and text mining for a theme inputted by accessing the database A decision support system by means of text mining composed of
The text mining device is:
Accessing the database, collecting opinion text groups in the database that are pre-classified as positive or negative opinions related to the input theme, and noun phrases and verb phrases appearing in the opinion text groups A point extraction means for extracting as a plurality of points related to the theme;
For each of the plurality of issues, a uniqueness calculating means for calculating a uniqueness representing a proportion of opinions having either affirmative or negative status with respect to the issue from the opinion text group including the issue;
For each of the plurality of issues, in the frequency of one position of the positive or negative in the feedback text group, said calculated by determining the ratio of the frequency of occurrence of the issue of the standpoint of the one Importance calculating means for calculating the importance of the opinion with respect to the issue;
Related word extracting means for extracting content words including nouns, verbs and adjectives that appear redundantly from the opinion text group including the argument as a related word related to the argument;
Selecting a plurality of positive or negative representative opinions for each of the plurality of issues by calculating the appearance frequency of the content word, scoring the appearance frequency of the selected plurality of representative opinions, Representative opinion selection means for ranking and outputting the plurality of representative opinions in descending order;
On the two-dimensional surface of the uniqueness and the importance, interface means for outputting image data in which the plurality of issues, the uniqueness for each issue, the importance, the related word, and the representative opinion are plotted;
A decision support system based on text mining characterized by having the following means.   The database includes a web search server having a browser for searching for data on multimedia connected to the Internet, and a database server for collecting and storing the searched data. Decision support system.   2. The decision support system according to claim 1, wherein the uniqueness calculated by the uniqueness calculating means is calculated by obtaining a probability of an opinion having an affirmative position with respect to each of the issues.   2. The decision support system according to claim 1, wherein the representative opinion selection means includes means for morphological analysis of all of the opinion text groups including the argument for each position opposite to approval of the argument.

Images