JP4911599B2 - Reputation information extraction device and reputation information extraction method - Google Patents

Description

  The present invention relates to a reputation information extracting method and apparatus using a computer that extracts reputation information published on a network for a predetermined target.

Unlike the conventional mass media and books, the spread of the Internet has made it easier for individuals to send information. Along with this, various opinions have been expressed, but uncertain information and information that damages honor (hereinafter referred to as reputation information) are easily transmitted.
And because there are so many websites, it's difficult to find them.

  On the other hand, the impact is serious for those who have received such reputation information. For example, when a user writes a product defect on a bulletin board or homepage on the Internet, even if the defect is misunderstood by the user, the reader will be impressed as if the product is true. .

In particular, since information retrieval is easy on the Internet, the fact that reputation information is circulated when a person who wants to purchase a product examines the product poses a serious obstacle to the economic activity of the company.
In fact, the destruction of corporate images and attacks on specific individuals have already occurred on the Internet, which has become a major social problem.

  As a method of dealing with this problem, it is necessary to manually search for articles related to self and find inappropriate ones. However, posting on the Internet is made every day, and real-time discovery is extremely difficult. If not discovered quickly, inappropriate information is exposed to many readers, making it more difficult to recover lost trust.

In this regard, the conventional manual method is extremely laborious and expensive to discover, and can be monitored on very well-known websites, and other flooded information is virtually silent. It can be said that there is.
Especially for small businesses and individuals, the discovery is almost impossible.

As a method for automatically finding such reputation information, the following techniques are disclosed.
First, the technology of Patent Document 1 acquires and stores Web pages on the Internet, analyzes the stored Web pages, divides them into block units, and determines whether or not each block unit includes a predetermined keyword. Then, when a keyword is included, reputation information for each predetermined keyword is extracted. Furthermore, after weighting each reputation information, it is proposed to calculate the importance of the reputation information based on the appearance status of the company product name and the other company's product name on the Web page.

  Next, in the technique of Patent Document 2, when a report output request including a keyword is transmitted from the user computer to the analysis server, the analysis server acquires a negative expression word, stores the stored word corresponding to the keyword, and acquires The database is queried for one or more pieces of text information in which the negative expression words appear simultaneously. The analysis server obtains a query result from a database and creates an analysis report that is arranged in a table or graph format.

JP 2004-70405 A JP 2005-63242 A

Any of the above prior arts is characterized in that, for example, when a keyword such as a company name and a word serving as reputation information are recognized, it is extracted as reputation information. For example, a rule-based extraction method is used in which a site is extracted when characters such as A product name and “defect” appear together.
In this method, extraction can be performed if a word of a negative expression defined in advance appears, but it cannot be extracted if the expression changes even a little. Considering that the reputation information is transmitted by various expression methods, it is difficult to accurately extract the reputation information by such conventional techniques.

  If you try to extract without omission, you have to register a very large number of words. In that case, there is a problem that a lot of information that is not reputation information is extracted. It will take.

  In addition, even if there is an expression of “defect”, if the manufacturer actually recalls it, if it is true, it is not reputation information, but the conventional technology effectively excludes information based on such facts. I can't.

  The present invention was created in view of the above-described problems of the prior art, and contributes to a technique for effectively determining whether or not the information is reputation information from information disclosed on the network, and particularly to the determination. An object of the present invention is to provide a technique for making a judgment by using a plurality of features in a composite manner.

In order to solve the above problems, the present invention provides the following reputation information extraction device.
The invention described in claim 1 is a reputation information extraction device using a computer that extracts reputation information published on a network for a predetermined target, and is disclosed from one or more server devices on the network. Data collecting means for receiving collected data and storing them as collected data in the collected data storage means, a feature table storage means for storing a feature table including at least a word or a set of words as a feature, and the feature table Referring to the feature extraction means for extracting features from the collected data and the input of one or a plurality of features, refer to the machine learning result data stored in the learning result storage means, Machine learning determination means having a predetermined machine learning module for determining whether or not the feature extracted by the feature extraction means is the machine Input to the learning determination means to obtain a determination result as to whether or not the collected data is reputation information, and at least a part of the collected data determined to be reputation information, or the name or network address of the server device to which it is disclosed Or reputation information output means for outputting at least one of the file information of the collected data.

The reputation information extraction device has a classification vocabulary table in which the meaning of words is classified as a semantic class using a code and each semantic class is assigned to a plurality of words, and the semantic class is used as a feature. It is included in the feature table, and the feature extraction means refers to the feature table and extracts a semantic class of a word included in the collected data.

According to the second aspect of the present invention, the reputation of the information extracted from the name or network address of the server device on the network or the file information of the collected data is expressed numerically by the reputation information extracting device. Including an information source reliability database to represent, and including the reliability in the feature table as a feature, and the feature extraction means refers to the feature table and extracts a numerical value of the reliability related to the collected data. Features.

According to a third aspect of the present invention, a reputation information extracting device includes data reliability value evaluation means, and the data reliability value evaluation means includes the collected data, the author of the collected data, or a stored server. An evaluation data extraction unit that extracts evaluation data that matches at least one of the device name or network address or the file information of the collected data from a server device on the network or an evaluation database that is stored in advance, and the collection An evaluation factor table storage unit for storing at least one of a positive factor that is a factor that increases the reliability of data and a negative factor that is a factor that decreases the reliability of the collected data; When the feature extraction unit for evaluation extracted from the data for evaluation and one or more features are input, the machine learning result stored in the learning result storage unit An evaluation machine learning determination unit including a predetermined machine learning module that performs classification according to the reliability of the evaluation data with reference to the data, and the features extracted by the evaluation feature extraction unit The classification result of the evaluation data obtained by inputting to the evaluation machine learning determination unit is output as the reliability of the collected data, and the feature extraction means extracts the reliability value of the collected data It is input to the machine learning determination means together with the feature, and a determination result as to whether or not it is reputation information for the collected data is obtained.

According to a fourth aspect of the present invention, the reputation information extracting device comprises ground information confirmation means, and the ground information confirmation means is the name or network address of a server device that discloses reliable information, or the reliable information Similarity in which the basis information database that defines at least one basis information source of information file information and the data published from the basis information source defined in the basis information database are acquired, and the topic is similar to the collected data A similarity determination unit for determining whether or not data is included, and the determination result in the similarity determination unit is input to the machine learning determination unit together with the features extracted by the feature extraction unit, and the collected data is A determination result of whether or not the information is reputation information is obtained.

The present invention can also provide the following reputation information extraction method.
That is, the invention described in claim 5 is a reputation information extraction method using a computer that extracts reputation information published on a network for a predetermined object, and the data collection means of the computer is connected to the network. A data collection step of receiving data published from one or more server devices and storing the data as collected data in the collected data storage means, and a feature table storing at least a feature table or a feature table including a set of words A feature extraction step of extracting a feature from the collected data by referring to the feature table, and a machine learning determination unit of a computer provided with a predetermined machine learning module. Using the extracted features, refer to the machine learning result data stored in the learning result storage means. , A machine learning determination step for determining whether or not the information is reputation information for a predetermined target, at least part of the collected data determined by the computer reputation information output means as reputation information, or the name of the server device to which it is disclosed Or a reputation information output step of outputting at least one of a network address or file information of the collected data.

  In the above configuration, the meaning of a word is classified as a semantic class using a code, and a classification vocabulary table in which each semantic class is assigned to a plurality of words is included in the feature table as a feature. In addition, in the feature extraction step, the feature extraction means refers to the feature table and extracts a semantic class of a word included in the collected data.

The invention according to claim 6 is an information source reliability database that represents numerically the reliability of information disclosed therein with respect to the name or network address of the server device on the network or the file information of the collected data. And including the reliability in the feature table as a feature, and in the feature extraction step, the feature extraction means refers to the feature table and extracts a numerical value of the reliability related to the collected data. Features.

The invention according to claim 7 has a data reliability evaluation step at any time after the data collection step of the reputation information extraction method and before the machine learning determination step, and the data reliability evaluation In the step, the data extraction unit for evaluation in the data reliability value evaluation means of the computer uses the collected data, the author of the collected data, or the name or network address of the stored server device, or file information of the collected data An evaluation data extraction process step for extracting evaluation data that matches at least one of the above from a server device on the network or an evaluation database stored in advance, an active factor that is a factor that increases the reliability of the collected data, and Stores at least one of the negative factors that reduce the reliability of the collected data An evaluation factor table storage unit, and an evaluation feature extraction unit in the data reliability value evaluation unit extracts the factor as a feature from the evaluation data, an evaluation feature extraction process step, and a data reliability value evaluation An evaluation machine learning determination unit having a predetermined machine learning module in the means uses the features extracted in the evaluation feature extraction processing step, refers to the machine learning result data stored in the learning result storage unit, Including the evaluation machine learning determination processing step for performing classification according to the reliability of the evaluation data, and outputting the evaluation data classification result as the reliability of the collected data, and in the machine learning determination step, The reliability value of the collected data is input to the machine learning determination unit together with the features extracted by the feature extraction unit, and a determination result as to whether the collected data is reputation information And wherein the get.

The invention according to claim 8 is the reputation information extraction method, wherein the clustering processing means of the computer, for at least one of the reputation information data and the related information data, the author or content included in any of the data, A clustering process step of performing a clustering process according to a predetermined clustering equation, wherein at least one of the reputation information data and the related information data in the clustered state is output in the output step.

The present invention has the following effects by providing the above configuration.
That is, according to the present invention, reputation information can be extracted using various features, and therefore it is possible to determine whether or not the reputation information is in consideration of complex elements that are impossible by hand.
Also, by using a computer, it is possible to search for reputation information quickly and comprehensively from a vast amount of information distributed on the network, so that damage caused by reputation information can be minimized.

  Hereinafter, embodiments of the present invention will be described based on examples shown in the drawings. The embodiment is not limited to the following.

FIG. 1 is an overall configuration diagram of a reputation information extracting device (1) (hereinafter referred to as this device) according to the present invention. The present invention can be easily realized by a known personal computer, and each step of the present invention is executed and processed by a CPU (10) that controls arithmetic processing, machine learning, text processing, and the like. As is well known, the CPU (10) operates in cooperation with a memory (not shown), and in addition to input means such as a keyboard and mouse (11), a monitor (12) for displaying output results, an external storage such as a hard disk. A device (13) is provided.
In addition, a network adapter (14) connected to a network such as the Internet is provided as data acquisition input means for acquiring text data.

The CPU (10) includes a data collection unit (100), a feature extraction unit (101), a machine learning determination unit (102), and a reputation information output unit (103).
And the program described by the well-known programming language operates CPU (10) and the hardware linked with it, The function of each part (100)-(103) demonstrated below is implement | achieved.

Hereinafter, each processing of the present invention will be described in detail with reference to a processing flowchart shown in FIG.
First, the data collection unit receives data published by each server device from a large number of server devices (20) installed on a network such as the Internet (21). (Data collection step: S1)
Specifically, text data (for example, text data named a1.txt) published on the web server A and data including a display format described in HTML (HyperTextMarkup Language) (for example, the name a2.html) Data).

As is well known, it is possible to collect a large amount of data published on the Internet using an automatic patrol robot engine. For example, a method of sequentially acquiring data by tracing hyperlinks is performed. . Alternatively, a method may be used in which a predetermined server is designated in advance and acquired in order according to the directory structure.
Data collection performed in the present invention can be performed by any method used in a search engine or the like.
The collected data is stored in the hard disk (13) which is a data storage means.

Next, a feature extraction unit (101) extracts features used for determining whether or not the information is reputation information in a machine learning determination unit (102) described later from the collected data. (Feature extraction step: S2)
At this time, a feature table is provided in the hard disk (13), and the feature extraction unit (101) extracts the features defined there.

For example, as a feature, information on what word or set of words (hereinafter collectively referred to as a word) is included, or information on whether or not a word is included is used.
Usually, a feature is given in the form of a word, its part of speech, etc., but the feature table of the present invention also stores a morpheme dictionary necessary for feature extraction.

The feature used by this invention is illustrated. The features used in the present invention are not limited to the following.
First, the features defined in the feature table (131) are shown in Table 1.

  A word or a collective feature of a word is a feature that needs to be used at the minimum in the present invention, and there are several types as shown in Table 2. These are extracted by performing morphological analysis on the collected data. Morphological analysis can be easily performed by using Chasen (Non-Patent Document 1), which is a well-known morphological analysis module.

http://chasen.naist.jp/

  By using Chasen, part of speech information, morphological occurrence cost, pronunciation information, utilization type information to specify utilization type, utilization form information to specify utilization form, original form to specify headword (basic form) Information, other additional information (semantic information), compound word information, etc. can also be acquired.

In the above, the name of the target is the most important, and the word that is the target of the reputation information is the feature. That is, when the user designates “information communication research organization” as a target for extracting reputation information, the feature extraction unit (101) detects whether or not the same word is included based on the morphological analysis result.
When the target is specified in this manner, whether or not the target is included is a premise for extracting reputation information, and thus data not included may be excluded without performing machine learning determination.

However, since the reputation information is not always described with the full name and the official name, for example, when three of “information”, “communication”, “research”, and “mechanism” are included as features, it is reputation information. Since the possibility cannot be denied, these may be used as features in machine learning determination.
As a result, in the case of the rule base, it was impossible to extract anything other than the defined words and phrases. On the other hand, in the method of the present invention, the possibility of reputation information is extracted from the combinations of the included words and phrases. Will be able to.

Next, the content of the reputation is the content of the concrete reputation, and is a phrase that is defined as a negative expression even in the rule base, such as “Lost”.
Of course, according to the present invention using machine learning determination, a plurality of words and phrases are also extracted as features. For example, if only “loss” is given, the possibility that the information is reputation information is 0.5. On the other hand, if “according to the report” “exceeding loss” is extracted, the probability is 0.1, and “according to rumors” “exceeding loss” “probably” is extracted The possibility is 0.9, so that a precise determination that cannot be obtained by the rule base can be performed.

  As other features, codes or words that are useful for determining the reliability of information can be used. If an emoticon (a code that expresses a facial expression by a combination of ASCII characters) is included as shown, this suggests that the information is not official information such as a press release or news site. Also, words that indicate the type and attributes of the website, such as “Angra”, and words that suggest personal feelings, such as “Feeling”, can be used as features. .

In addition, use the name of the server (name assigned to the device, domain name, host name, etc.) and file information (file name, directory name, extension, etc.) as features. You can also.
As illustrated, if a word such as bbs is included, it indicates that it is a bulletin board on the network, and there is a high possibility that it is not public information. In addition, when “~” (tilde) is included, there is a possibility that it is a homepage established by an individual, and this is also suitable as a feature used for determining whether or not it is reputation information.

In addition, the rate of occurrence of unknown words is generally relatively low in morphological analysis of official documents, and conversely, in terms of information that includes reputation information, slang terms, especially new expressions that have recently become widely used on the Internet, are seen. There are many cases.
Utilizing such characteristics, the present invention also proposes that an unknown word is determined from a result that cannot be analyzed because it is not registered in the dictionary when morphological analysis is performed, and the result is used as a feature.

In this case, whether or not an unknown word is included may be used as a feature, and the proportion of unknown words (such as some percent of all words) may be used as a feature.
By using information related to an unknown word as a feature, for example, for information that uses a slang word such as “Kakiko” (meaning writing on a bulletin board), it is determined that the information is likely to be reputation information. Reflected.

Next, formatting information can also be used as a feature in the present invention. Specific examples are shown in Table 3.

  The format setting information has, for example, a format specified by a tag in HTML as a feature. Here, when the background is black and the text color is red, the web page with such a color scheme is often not a website that sends official information, and such formatting information is used as a feature. This also contributes to accurate extraction of reputation information.

Semantic classes can also be used as features.
Here, the semantic class is a classification based on semantic similarity of words, and a classification vocabulary table classified according to the semantic class is created as shown in Table 4.

  In the present invention, the classification vocabulary table is stored in the feature table (131). The classification vocabulary table is generally a table in which words are arranged based on meaning, and a number called a classification number is assigned to each word. This 10-digit classification number indicates a 7-level hierarchical structure, with the top 5 levels represented by the first 5 digits of the classification number, the 6th level is the next 2 digits, and the lowest level is the last 3 levels. It is expressed in digits.

  By using such a classification vocabulary table, words having similar meanings can be extracted in a lump by limiting them with the upper digits of the classification number. That is, the feature extraction unit (101) inquires the classification number of the word subjected to the morphological analysis based on the classification vocabulary table, and extracts the classification number as the feature. At this time, it is possible to cover a wide range of words having similar meanings by making the upper 5 digits or 7 digits of the classification number a feature.

Table 5 shows examples of semantic class features used in the present invention.

  The feature used in the present invention may be a reliability value based on an information source reliability database as shown in Table 6. Again, the database is stored in the feature table (131), and the feature extraction unit (101) extracts reliability by referring to the database.

Specifically, the feature extraction unit extracts the domain name of the server device of the collected data, refers to the information source reliability database, and uses the reliability value in the machine learning determination unit (102). For example, if the domain name of the collected data is www.asahi.com, and if a prominent news site is the source of information, it would be information that is likely to be popular. However, it is a true news report and cannot be said to be reputable information. In this way, the reliable information source reliability database is provided in advance, and the reliability value defined there is used as a feature, thereby contributing to accurate reputation information extraction.
Further, the IP address and file information may be defined in the information source reliability database as features. Table 7 shows an example of information source reliability.
In the present invention, the reliability of the information source can also be automatically evaluated, which will be described later.

The present invention is characterized by using various features described above. Since such features cannot be reliably recognized as reputation information by one by one, conventional rule-based extraction of reputation information cannot make a determination taking these features into account.
In the present invention, each feature is evaluated with an appropriate weight by using machine learning, and it is determined whether or not it is reputation information.

In the present invention, the result of machine learning as described above is stored as machine learning result data (132) of the hard disk (13) and used for determination in the machine learning determination unit (102).
In the machine learning method, a learning process in a known machine learning module and a solution estimation process using the learning process are integrated. In the implementation of the present invention, the learning process is not necessarily required, and the machine learning result data (132) may already be provided. However, in consideration of such features of machine learning, known methods are briefly described for both processes. To do.

  The machine learning method prepares many sets of problem-solution pairs, learns them, learns what kind of solution is the problem, and uses the learning results to create a new problem. This is a method that makes it possible to guess the solution (for example, see Non-Patent Document 2 to Non-Patent Document 4 below).

Masaki Murata, Language processing based on machine learning, Faculty of Science and Engineering, Ryukoku University. Invited lecture. 2004. http://www2.nict.go.jp/jt/a132/members/murata/ps/rk1-siryou.pdf Japanese-English translation of tense aspect modality using support vector machine, Maki Murata, Ma Aoi, Kiyotaka Uchimoto, Hitoshi Isahara, IEICE Language Understanding and Communication Study Group NLC2000-78, 2001. CRL's efforts in the SENSEVAL2J dictionary task, Masaki Murata, Masao Uchiyama, Kiyotaka Uchimoto, Ma Aoi, Hitoshi Isahara, IEICE Language Understanding and Communication Study Group NLC2001-40, 2001.

In order to convey the problem situation to the machine, what kind of problem is required, features (elements constituting the problem with information used for analysis) are required. The problem is expressed by the feature. For example, in the problem of estimating the tense of Japanese sentence ending expressions,
Problem: "He speaks." --- Solution "Current"
Is given as an example, “He speaks.” “Speaks.” “Speaks.” “Su.” “.”.

  In other words, the machine learning method prepares many sets of feature set-solution pairs, learns with it, learns what kind of feature set the solution will be, and uses the learning results. This is a method of extracting a set of features from a new problem and inferring a solution in the case of the feature.

FIG. 3 is a configuration diagram of an apparatus for executing the machine learning process. The CPU (10) includes a solution-feature pair extraction unit (141) and a machine learning unit (142) as a previous stage to be processed by the machine learning determination unit (102). Here, the machine learning process obtains the classification result (solution) as to how the distributed text data is classified as shown in FIG.
As a machine learning method in the machine learning unit (142), for example, a k neighborhood method, a simple Bayes method, a decision list method, a maximum entropy method, a support vector machine method, or the like is used.

 The k-nearest neighbor method is a method for obtaining a classification destination (solution) by using the most similar k cases instead of the most similar case, and by majority decision of these k cases. k is a predetermined integer number, and generally an odd number between 1 and 9 is used.

 The Simple Bayes method is a method of estimating the probability of each classification based on Bayes' theorem and determining the classification having the highest probability value as a classification destination.

 In the Simple Bayes method, the probability of outputting the classification a in the context b is given by the following formula 1.

Here, the context b is a set of features f _j (∈F, 1 ≦ j ≦ k) set in advance. p (b) is the appearance probability of context b. Here, it is independent of the classification a and is not calculated because it is a constant. P (a) (where P is a tilde at the top of p) and P (f _i | a) are the probabilities estimated from the teacher data, respectively, and the probability f of class a, and the feature f for class a means the probability of having _i . When the value obtained by performing maximum likelihood estimation as P (f _i | a) is used, the value often becomes zero, and it may be difficult to determine the classification destination because the value of Equation 2 is zero. Therefore, smoothing is performed. Here, the smoothing using the following equation 3 is used.

Here, freq (f _i , a) means the number of cases having the feature f _i and the classification a, and freq (a) means the number of cases having the classification a.

  The decision list method uses features and combinations of classification destinations as rules, stores them in the list in a predetermined priority order, and when input to be detected is given, from the highest priority in the list This is a method in which input data is compared with the feature of the rule, and the classification destination of the rule having the same feature is set as the classification destination of the input.

In the decision list method, the probability value of each classification is obtained using only one of the features f _j (εF, 1 ≦ j ≦ k) set in advance as a context. The probability of outputting classification a in a context b is given by

(Equation 4)
p (a | b) = p (a | fmax)
However, fmax is given by the following equation (5).

P (a _i | f _j ) (where P is a tilde at the top of p) is the rate of appearance of classification a _i when feature f _j is in the context.

In the maximum entropy method, when F is a set of features f _j (1 ≦ j ≦ k) set in advance, the maximum number 7 representing entropy is satisfied while satisfying the predetermined conditional expression (formula 6). This is a method of obtaining a probability distribution p (a, b) for the classification and obtaining a classification having the largest probability value among the probabilities of the respective classifications determined according to the probability distribution.

However, A and B mean a set of classification and context, and g _j (a, b) is a function that is 1 if the context b has a feature f _j and the classification is a, and 0 otherwise means. P (a _i | f _j ) (where P is a tilde at the top of p) means the rate of appearance of (a, b) in the known data.

 Equation 6 is to obtain the expected value of the frequency of the output and feature pair by multiplying the probability p and the function g which means the appearance of the pair of output and feature, With the restriction that the expected values calculated based on the probability distribution obtained on the left side are equal, entropy maximization (smoothing of the probability distribution) is performed to obtain the probability distribution of the output and the context. Details of the maximum entropy method are described in Non-Patent Document 5 and Non-Patent Document 6 below.

Eric Sven Ristad, Maximum Entropy Modeling for Natural Language, (ACL / EACL Tutorial Program, Madrid, 1997 Eric Sven Ristad, Maximum Entropy Modeling Toolkit, Release 1.6beta, (http://www.mnemonic.com/software/memt,1998)) Is a method for classifying data consisting of

  FIG. 5 shows the concept of margin maximization in the support vector machine method. In FIG. 5, a white circle means a positive example, a black circle means a negative example, a solid line means a hyperplane that divides the space, and a broken line means a surface that represents the boundary of the margin area. 5A is a conceptual diagram when the interval between the positive example and the negative example is small (small margin), and FIG. 5B is a conceptual diagram when the interval between the positive example and the negative example is wide (large margin). is there.

  At this time, if the two classifications consist of positive and negative examples, the larger the interval (margin) between the positive and negative examples in the learning data, the less likely it is to make an incorrect classification with open data. As shown in FIG. 5B, a hyperplane that maximizes this margin is obtained, and classification is performed using it.

  Basically, it is as described above, but usually, an extension of the method that a small number of cases may be included in the inner area of the margin in the training data, or an extension that makes the linear part of the hyperplane nonlinear ( The one with the introduction of the kernel function is used.

  This extended method is equivalent to classification using the following discriminant function (Equation 8), and the two classes can be discriminated depending on whether the output value of the discriminant function is positive or negative.

Where x is the context (set of features) to be identified, and x _i and y _j (i = 1, ..., l, y _j ∈ {1, -1}) are the context of the learning data and the classification destination Means the function sgn
sgn (x) = 1 (x ≧ 0)
-1 (otherwise)
Further, each α _i is for the case where the number 9 is maximized under the constraints of the numbers 10 and 11.

  The function K is called a kernel function, and various functions are used. In this embodiment, the following polynomial is used.

(Equation 12)
K (x, y) = (x ・ y + 1) d

C and d are constants set experimentally. For example, C was fixed at 1 throughout all treatments. Moreover, d is trying two kinds of 1 and 2. Here, x _i satisfying α _i > 0 is called a support vector, and the portion taking the sum of Expression 8 is usually calculated using only this case. That is, only actual cases called support vectors are used for actual analysis.

  Details of the extended support vector machine method are described in Non-Patent Document 7 and Non-Patent Document 8 below.

Nello Cristianini and John Shawe-Taylor, An Introduction to SupportVector Machines and other kernel-based learning methods, (Cambridge University Press, 2000) Taku Kudoh, Tinysvm: Support Vectormachines, (http://chasen.org/~taku/software/TinySVM/, 2002)

  The support vector machine method handles data with two classifications. Therefore, when dealing with cases where the number of classifications is 3 or more, usually, a method such as the pair-wise method or the one-VS rest method is used in combination.

  The pairwise method generates all pairs (n (n-1) / 2) of two different classification destinations in the case of data having n classifications, and the binary classifier determines which is better for each pair. In other words, it is a method of obtaining a classification destination by a majority decision of classification destinations based on n (n-1) / 2 binary classifications, which is obtained by a support vector machine method processing module.

  For example, when there are three classification destinations, a, b, and c, the one-VS rest method generates three sets of classification destination a and other, classification destination b and other, classification destination c and other, The learning process is performed on the set of the support vector machine method. Then, in the estimation process based on the learning result, the learning results of the three sets of support vector machines are used. See how the three support vector machines are estimated as candidates to be estimated. Of the three support vector machines, it is the non-other classification target and the separation plane of the support vector machine. This is a method for obtaining a classification destination of a thing farthest from the object. For example, when a candidate is farthest from the separation plane in the support vector machine created by the learning process of “classification destination a and others”, the candidate classification destination is estimated as a.

  Then, the machine learning unit (142) uses the machine learning method (142) as a method of machine learning to determine the degree of the solution (classification destination) that is likely to be a reputation as to whether or not it is reputation information estimated by the machine learning determination unit (102). It depends on the various methods used.

  For example, in the embodiment of the present invention, when the machine learning unit (142) uses the k-nearest neighbor method as a machine learning method, the machine learning unit (142) is extracted from the cases of the teacher data examples. Define the similarity between cases based on the ratio of overlapping features in the set of features (the ratio of how many of the same features), and machine learning results using the defined similarities and cases as learning result information This is stored in the data (132).

  Then, the machine learning determination unit (102) refers to the probability that the data extracted by the feature extraction unit (101) is reputation information defined in the machine learning result data (132) and the feature. Then, the k features are selected from the examples of the machine learning result data (132) in the descending order of the possibility that the data is the reputation information, and the classification destination of whether or not the reputation information is determined by the majority decision with the selected k features. Is estimated as a solution. That is, in the machine learning determination unit (102), the degree of what kind of solution (classification destination) is likely to be obtained for each extracted data is determined by the number of votes of majority vote with the selected k features, The number of votes acquired by the classification “reputation information”.

  When the simple Bayes method is used as a machine learning method, the machine learning unit (142) uses machine learning result data for a case of teacher data, using a combination of the solution of the case and a set of features as learning result information. (132). Then, the machine learning determination unit (102), when the data collection unit (100) extracts the data, based on the set of the learning result information of the machine learning result data (132) and the feature set, For the feature set acquired by the feature extraction unit (101) based on Bayes' theorem, the probability of each classification related to whether it is reputation information is calculated, and the classification with the largest probability value is Estimate the feature classification (solution) for the data. That is, in the machine learning determination unit (102), the probability of being a certain solution in the case of a set of features about the extracted data is set to the probability of being classified into each category, in this case, the classification is “reputation information”. Probability.

  When the decision list method is used as the machine learning method, the machine learning unit (142) creates a list in which rules of features and classification destinations are arranged in a predetermined priority order for the example of the teacher data. (132). Then, when the data collection unit (100) extracts the data, the machine learning determination unit (102) identifies the features of the extracted expression pair candidates in descending order of priority in the list of the machine learning result data (132). And the feature of the rule are compared, and the classification destination of the rule having the same feature is estimated as the candidate classification destination (solution). That is, in the machine learning determination unit (102), the degree of easiness to be a solution for the extracted data in the case of the set of features is determined by a predetermined priority or a numerical value, scale corresponding thereto, The priority is in the list of probabilities of being classified as “some”.

  When the maximum entropy method is used as the machine learning method, the machine learning unit (142) identifies a class that can be a solution from the example of the teacher data, and satisfies the predetermined conditional expression and expresses the entropy. A probability distribution composed of a set of features when maximizing and a binomial classification that can be a solution is obtained and stored in the machine learning result data (132). Then, when the data collection unit (100) extracts the data, the machine learning determination unit (102) uses the probability distribution of the machine learning result data (132) to determine the set of features of the extracted data. The probability of the classification that can be the solution is obtained, the classification that can be the solution having the largest probability value is specified, and the specified classification is estimated as the candidate solution. That is, in the machine learning determination unit (102), the degree of the likelihood of becoming a solution in the case of the set of features of the extracted data is the probability of each classification, in this case, the classification is “reputation information” Probability.

  When the support vector machine method is used as the machine learning method, the machine learning unit (142) identifies a class that can be a solution from the example of the teacher data (140), and divides the class into a positive example and a negative example. Then, in a space whose dimension is a set of case features according to a predetermined execution function using a kernel function, the interval between the positive example and the negative example is maximized, and the positive example and the negative example are hyperplanes. The hyperplane to be divided by is obtained and stored in the machine learning result data (132). Then, when the data collection unit (100) extracts the data, the machine learning determination unit (102) uses the hyperplane of the machine learning result data (132), and the feature set for the extracted data is over. In the space divided by the plane, it is identified whether the space is on the positive example side or the negative example side, and the classification determined based on the identified result is estimated as the candidate solution. That is, in the machine learning determination unit (102), the degree of easiness to be a solution in the case of the set of features of the extracted data is determined from the distance from the separation plane to the space of the positive example (data which is reputation information). The size of More specifically, when data that is reputation information is a positive example and data that is not reputation information is a negative example, the data located in the space on the positive example side with respect to the separation plane is determined as “data that is reputation information”. The distance from the separation plane of the case is set as the degree of the reputation information of the data.

Furthermore, in the present invention, a method using a known neural network or a method using multiple regression analysis can be used as a machine learning method.
For example, if there are two types of classification to be obtained, multiple regression analysis can be used. The method of executing the multiple regression analysis on a computer is detailed in Non-Patent Document 9.

"Time series analysis and forecasting with Excel", Chapter 3, Ohm

  In the case of multiple regression analysis, as many explanatory variables x as the number of features are prepared, and the presence or absence of the features is represented by 1,0 as the value of the explanatory variable x. The objective variable (explained variable) may be obtained with a value of 1 for a certain classification and a value of 0 for another classification.

As described above, the present invention generates machine learning result data (153) by a machine learning module (FIG. 3) having any known machine learning method, and then the machine learning determination unit (102) It is judged accurately whether it is reputation information. (Machine learning determination step: S3)
Whether or not it is reputation information may be output as either “reputation information” or “not reputation information” by the machine learning method as described above, or “probability of reputation information” is output. Sometimes it is done. The “probability of reputation information” may be output together with the probabilities in descending order. Further, it may be output together with a format for indicating the probability, for example, a character color, a character size, or a mark indicating the probability.

From the reputation information output unit (103), either “reputation information” or “not reputation information” may be output as reputation information (reputation information output step: S4), or the probability of reputation information may be determined. You may output as it is.
Furthermore, a threshold value set by the user or a predefined threshold value may be used, and when the probability exceeds the threshold value, “reputation information” may be output.

As an output method, in addition to displaying from the monitor (12), a method of transmitting the result from the network adapter (14) to another terminal device, or a method of storing as a reputation information extraction database in the hard disk (13) may be used.
At that time, at least one part of the collected data, or the name or network address of the server device to which the data is disclosed, or the file information of the collected data is output together with the result. In particular, a word or the like that is determined as the reputation information may be output.

In the above description, the target name is used as one of the features. However, the target name is not necessarily specified by the user, and can be automatically defined.
That is, the extraction of reputation information executed by the present invention uses proper nouns such as company names, product names, and personal names, and these are used for a site designated by the user or a keyword designated by the user as a known search engine. It can be extracted from the data contained in the site searched by typing.

For this purpose, the CPU (10) of the present invention includes a determination target noun extraction unit (not shown), and acquires data from a site designated via the network adapter (14) based on the site information stored in the hard disk in advance. . Alternatively, it accepts a keyword from the user via the keyboard (11), outputs the keyword to the search site based on the search site information stored in the hard disk, and obtains a site search result from the search site, Get data from the site.
A proper noun is extracted from the acquired data. In the present invention, general nouns may be used instead of proper nouns.
For this extraction, a specific expression can be automatically extracted from the data and used as a feature by using the following specific expression extraction technique.

Hereinafter, an example of a general technique for extracting a specific expression will be described.
(1) Method using machine learning There is a method of extracting a specific expression using machine learning (for example, see Non-Patent Document 10 below).

Masayuki Asahara, Yuji Matsumoto, Utilization of Redundant Morphological Analysis in Japanese Named Expression Extraction Information Processing Society of Japan Natural Language Processing Study Group NL153-7 2002

First, for example, the sentence “Japan's prime minister is Mr. Koizumi” is divided into each character, and correct characters such as B-LOCATION and I-LOCATION are attached to the divided characters as follows. To set the correct answer. The first column below is each divided character, and the correct tag of each character is the second column.

Day B-LOCATION
I-LOCATION
O
Neck O
Phase O
Is O
Small B-PERSON
Izumi I-PERSON
O
N
At O
O
. O

In the above, B-??? is a tag that means the beginning of the type of proper expression below the hyphen. For example, B-LOCATION means the beginning of a unique expression called place name, and B-PERSON means the beginning of a unique expression called person name. Also, I-??? is a tag that means other than the beginning of the type of proper expression below the hyphen, and O is other than these. Therefore, for example, the character “day” is a character that corresponds to the beginning of the unique name “place name”, and the character “book” is the unique name “place name”.

  In this way, the correct answer of each character is set, learned from such data, this correct answer is estimated with new data, and from this correct answer tag, the beginning of each proper expression and how far it is. Is recognized and the proper expression is estimated.

When learning from the correct data set for each character, the system uses various information in the form of features. For example,
Day B-LOCATION
Part of
Japan-B Noun-B
Such information is used. Japan-B means the beginning of the word Japan, and noun-B means the beginning of the noun. For the recognition of words and parts of speech, for example, the morphological analysis by Chasen described above is used. The above-mentioned Chasen can also estimate the part of speech of each word, so if you enter "go to school" you get the following results.

School Gakkou School Noun-General
To He To Particle-Case particle-General
Go Iku Go Verb-Independence
EOS
In this way, each line is divided so that one word is included, and reading and part-of-speech information are given to each word.

  Note that, for example, in Non-Patent Document 10 described above, as features, characters themselves (for example, “small”), character types (for example, hiragana and katakana), part-of-speech information, tags, Information (for example, “B-PERSON” etc.) is used.

 Learning using these features. Investigate what features appear in the characters that estimate the tag and the surrounding characters, learn what features are likely to appear when the features appear, and use the learning results to create new data Perform tag estimation. For machine learning, for example, a support vector machine is used.

  In addition to the above-described method, there are various methods for extracting the proper expression. For example, there is a technique for extracting a specific expression using a maximum entropy model and a rewrite rule (see Non-Patent Document 11).

Uchimoto Kiyotaka, Ma Aoi, Murata Maki, Osaku Hiromi, Uchiyama Masao, Isahara Hitoshi, Entity Expression Extraction Based on Maximum Entropy Model and Rewriting Rules, Journal of the Language Processing Society, Vol. 7, No. 2, 2000

Also, for example, in the following Non-Patent Document 12, using a support vector machine,
It describes a technique for extracting expressions.

Hiroyasu Yamada, Taku Kudo, Yuji Matsumoto, Japanese Named Expression Extraction using Support Vector Machine, Journal of Information Processing Society of Japan, Vol.43, No.1 ", 2002

(2) A method using the created rule There is also a method of manually creating a rule to extract a specific expression.
For example,
A noun + “san” means a person
Name as noun + "Prime Minister"
Noun + “corporation” means company name + “town” means place name
For example, a noun + “city” is a place name.

  A unique expression is extracted by the above method, and a person name, a company name, etc., for example, can be extracted in the feature extraction unit (101) among the extracted expressions.

In addition to the above features, the present invention is characterized in that data related to the data collected by the data collection unit (100) is extracted.
As one example, a technique for evaluating the reliability of data is proposed. As shown in FIG. 6, the CPU (10) of the apparatus (1) includes an evaluation data extraction unit (104), an evaluation feature extraction unit (105), and an evaluation machine learning determination unit (106). The reliability of the data is determined, and the result is used as a feature in the machine learning determination unit (102).

FIG. 7 is a processing flowchart including a data reliability evaluation step (S5) according to the present embodiment.
In this configuration, the evaluation database (133) is stored in the hard disk (13) or stored in an arbitrary server on the network. The evaluation database (133) includes a large amount of data, and the evaluation data extraction unit (104) includes the data collected by the data collection unit, the author of the collected data, or a server to be stored. Evaluation data that matches at least one of the device name or network address and the file information of the collected data is extracted. (Evaluation data extraction step: S50)

That is, first, the evaluation data extraction unit searches for the creator from the collected data. For the search, for example, when the author is clearly indicated as “literal responsibility: ○ Ogawa ○ O”, “Ogawa Oo” is extracted. Alternatively, non-display author information included in HTML may be extracted.
Then, the evaluation data including “Ogawa ○ O” is extracted from the evaluation database (133).

  In addition to the author, the server device name or network address stored or the file data of the collected data may match the evaluation data. For example, when the collected data is data collected from the domain www.nhk.or.jp, the evaluation data published from the same domain is extracted.

  Then, an evaluation factor table (134) as shown in Table 8 is stored in the hard disk (13).

  Although the evaluation factor is not limited to the above, for example, when the expression “XX Co., Ltd. Representative Director ○ Ogawa ○ o” is included in the evaluation data, the work place of this author is “XX Co., Ltd.” In other words, it is extracted that the title is “representative director”. With regard to a work, if a library database is used as evaluation data, it can be detected if there is a work of “* kawa * o”. (Even if someone with the same name has the same name, the present invention is not evaluated to be definitely reliable with this, so it is not a serious problem.)

  Also, what topic the extracted evaluation data is can be extracted by a summarization technique to be described later. Furthermore, inappropriate words (such as the above-mentioned emoticons and emotional expressions) in the evaluation data can be extracted.

Such an evaluation factor is extracted from the evaluation data by the evaluation feature extraction unit (105) as a feature in machine learning. (Evaluation feature extraction step: S51)
Then, using these features, the evaluation machine learning determination unit (106) calculates the reliability of the evaluation data as either “reliable” or “unreliable” expressed in numerical values. (Evaluation machine learning determination step: S52)
For this determination, the machine learning result data (135) prepared for evaluation is used. The machine learning result data (135) is also generated by the above-described machine learning method.

According to the above configuration, it is possible to extract evaluation data and evaluate the reliability of the data collected by the data collection unit (100). In general, when information transmitted from a certain author or a certain website is highly reliable, it is considered that the information disclosed elsewhere is also highly reliable.
Therefore, in the present invention, the reliability determined for the evaluation data is input as a feature in the machine learning determination unit (102), thereby contributing to more accurate determination of reputation information.

  Further, in the present invention, the machine learning determination unit (102) may be input as a feature without being input, or may be output as related information at the time of output from the reputation information output unit (103).

  The above method extracts the evaluation data and uses the reliability of the evaluation data to determine the reputation information. This is further advanced and it is confirmed whether there is a similar description in the very reliable basis information. Technology can be provided.

  FIG. 8 shows a configuration related to the present technology. The basis for storing information similar to the data collected by the data collection unit (100) in the hard disk (13) in the CPU (10) of the apparatus (1) in advance. It is determined whether or not it exists in the ground information source included in the information database (136).

  In the basis information database (136), as shown in Table 9, at least one of the name or network address of the server device that can be the basis information and the file information of the reliable information is stored.

  As shown in FIG. 9, in the present embodiment, after the data collection step (S1), the ground information confirmation step (S6) is executed, in which the similarity determination unit (107) executes the similarity determination step (S61). To do. In accordance with the file information, domain name, site name, etc. included in the rational information database (136), the similarity determination unit (107) acquires data from each rational information source, and the data collected therein and the topic are common It is determined whether or not information is included.

Here, as a method for performing similarity determination, the following similar document search technique can be used.
First, after data collection, a method of extracting ground information data including a large number of word groups A constituting the data obtained by performing the collected data morphological analysis will be described.

(1) Explanation of basic method (TF / IDF method)
(Equation 13)
score (D) = Σ (tf (w, D) * log (N / df (w)))
Add by w ∈W
W is a set of keywords, tf (w, D) is the number of occurrences of w in the collected data, df (w) is the number of documents in which W appears in all documents, N is the total number of documents The document data having a high score (D) is output as similar basis information data.

(2) Explanation of Okapi weighting by Robertson et al.
This method is described in Non-Patent Document 13.

Murata Masaki, Ma Ao, Uchimoto Kiyotaka, Osaku Hiromi, Uchiyama Masao, Isahara Hitoshi "Information Retrieval Using Location Information and Field Information" Natural Language Processing (Journal of the Language Processing Society) April 2000, Vol. 7, No. 2 , p.141 to p.160 It is known that Equation 14 in Non-Patent Document 13 has good performance. The product of tf term and idf term before taking the product by Σ becomes Okapi's weighting method, and this value is used for the word weight.

Okapi formula (14)
score (D) = Σ (tf (w, D) / (tf (w, D) + length / delta) * log (N / df (w)))
Add by w ∈W

length is the length of data D, delta is the average length of data, and the length of data is the number of bytes of data, the number of words included in the data, or the like.

Further, the following information search can be performed.
(Okapi reference)
The Okapi equation and SMART equation disclosed in Non-Patent Documents 14 and 15 can also be used. As a more advanced information retrieval method, these Okapi SMART formulas may be used instead of just formulas using tf · idf.

S. E. Robertson, S. Walker, S. Jones, M. M. Hancock-Beaulieu, and M. Gatford Okapi at TREC-3, TREC-3, 1994 Amit Singhal AT & T at TREC-6, TREC-6, 1997

 In these methods, more accurate information retrieval can be performed using not only tf / idf but also the length of the article.

 In this method of extracting articles including more word groups A, Rocchio'sformula (Non-Patent Document 16) can be used.

J. J. Rocchio, Relevance feedback in information retrieval, The SMARTretrieval System, Edited by G. Salton, Prentice Hall, Inc., page 313-323, 1971

This method can be used instead of log (N / df (w))
(Equation 15)
{E (t) + k_af * (RatioC (t)-RatioD (t))} * log (N / df (w))
use.

E (t) = 1 (keyword from the original search)
= 0 (otherwise)
RatioC (t) is the appearance rate of t in data group B
RatioD (t) is the appearance rate of t in article group C
The score (D) is obtained by replacing log (N / df (w)) with the above equation, and the larger the value, the more the word group A is extracted.

 A set W of words w added when Σ of score (D) is added is both the original keyword and the word group A. However, the original keyword and the word group A should not overlap.

  Another method is to add Σ of score (D). The set W of words w is only the word group A. However, the original keyword and the word group A should not overlap.

  Here, the roccio formula is used in a complicated manner, but simply, the larger the sum of the number of occurrences of words in word group A, the more the word group A may be taken out, or Alternatively, an article having a larger difference in appearance of the word group A may be extracted as an article including more word groups A.

By the above method, an article including the word group A can be taken out and extracted as ground information data.
As one process in the ground information confirmation step (S6) in the present invention, there is a method by extracting an article including the word group A as described above.

Next, as another method in the ground information confirmation step (S6), a method for extracting similar articles in the collected data group B will be described.
Define the similarity between articles. Use tf / idf, okapi, or smart for this similarity. You can use two articles x and y that compare articles D and queries in tf / idf, okapi, smart, etc. And let the word included in both x and y be w.

  Create a vector with each word as a dimension and the score of each word as an element, change the vector of article x to a vector (vector _x) using the words contained in article x, and the vector of article y to article y The included words may be used as vectors (vector_y), and the value of the cosine (cos (vector_x, vector_y)) of the vectors may be used as the similarity of articles. Use tf / idf, okapi, or smart to calculate the score of each word.

  The expression after the Σ of those expressions is the expression for calculating the score. The value of the expression is the score for each word.

tf ・ idf tf (w, D) * log (N / df (w))
For okapi, tf (w, D) / (tf (w, D) + length / delta) * log (N / df (w))
Is the formula.

  Also, even in the extraction of articles containing more word groups A, the value of the cosine (cos (vector_x, vector_y)) of this vector is obtained, and an article having a larger word group A is judged to be an article containing more word groups A. Good. In this case, the word included in the word group A is used as a vector (vector_x), and the word included in the article is used as a vector (vector_y).

The similarity between the data group B and the ground information data x includes the following methods.
(1) Data that is most similar to the basis information data x in the data group B and a method that uses the similarity of the basis information data x as the similarity (2) Articles that are the least similar to the basis information data x in the data group B And a method of setting the similarity of the basis information data x as the similarity (3) a method of setting an average of the similarities of all articles in the data group B and the basis information data x as the similarity
Although other methods may be used, the similarity between the data group B and the basis information data x can be obtained in this way, and the article having a large similarity can be extracted as a similar article.

  As another method, a word that appears biased in the data group B is extracted by the previous method, and the score (D) based on the Rocchio's formula is calculated using the word, and the Score (D) Larger data may be extracted as similar data.

By this method, a similar article can be taken out and may be extracted as ground information data in the present invention.
Furthermore, the article may be extracted by executing both the above-described method of extracting an article including the word group A and the method of extracting a similar article described here.

Using the similar document extraction technique as described above, the similarity determination unit (107) in the present invention determines the similarity and inputs whether or not there is a similar document as a feature to the machine learning determination unit (102). To do.
When the similarity is calculated numerically, if there is a document with similarities greater than the value obtained by multiplying the similarity by p (p <1) in the ground information data, the collected data shall be grounded .

  Here, in the case of the method based on extraction of articles including the word group A, normalization is required. As normalization, it is assumed that there is a document having the same keyword group as the input keyword group, and the score of the document is calculated. Then, the normalized similarity can be calculated by dividing the similarity by the score. Note that the normalization method is arbitrary.

Similarity determination can also be performed using the paraphrasing technique disclosed in Non-Patent Document 17. In this method, first, a large number of synonym paraphrasing transformation rules are prepared, and the paraphrasing is performed using these rules. Paraphrasing with increasing similarity is performed, and the similarity is obtained between sentences having the highest similarity. The similarity can be calculated as a score, for example, how many keywords included in one sentence and the pattern appearing in the vicinity are extracted in the other sentence, but not limited to this. The similarity can be calculated by this method.
This paraphrase may be paraphrased for both sentences for which the degree of similarity is obtained.

  In this way, when the degree of similarity is calculated after paraphrasing, the sentences are in a similar state, and thus the degree of similarity between sentences can be calculated more accurately.

Masaki Murata and Hitoshi Isahara, “Unified Paraphrasing Model-Utilization of Scale-Based Deformation”, Natural Language Processing, Vol.11, No.5, p.113-133, Linguistic Processing Society of Japan, October 2004

(Method of using machine learning to extract related information)
In the above description, the related information is extracted by the related information extraction unit (110) of the CPU (10) regarding the collected data determined as the reputation information, the author, the content, the name or network address of the server device to be stored, and file information The related information is extracted based on the above.
In the present invention, it is also possible to determine the validity of extracting related information by using a machine learning model for extracting related information itself.

The CPU (10) includes a related information similarity calculation unit (not shown) having a method for calculating the similarity of the above article, and a related information evaluation unit (illustration) that evaluates the validity of the related information using a machine learning model. Do not).
The machine learning module included in the related information evaluation unit performs machine learning in the following manner and stores the result on the hard disk as machine learning result data.

For machine learning, a large amount of reputation information data collected by the data collection unit (100) and related information data collected by the related information extraction unit (110) are prepared and input to the machine learning module. Input data.
It is manually determined whether or not the related information data is really appropriate as the related information data, and the result is used as output data for learning.

At the same time, the related information similarity calculation unit calculates the reputation information data for learning and the related information similarity of the related information data for learning. The method for calculating the similarity is as described above. The similarity may be a binary result that is similar or not, or may be a numerical value indicating the degree of similarity.
Then, as features in machine learning, a word string constituting the reputation information data of the input data and a related information similarity are used together with a word string constituting the related information data.

The related information evaluation unit is provided, and the related information data extracted by the related information extraction unit (110) and the reputation information data at that time are input to determine whether the related information data is valid. According to the determination result, it is output from the reputation / related information output unit (103) as related information.
Any of the various methods described above can be used as the machine learning method in the machine learning module, and the judgment result at this time is output as “valid” or “invalid”, and the probability of being valid. May be output. In the former case, “valid” related information data may be output, and in the latter case, it may be output when the probability exceeds a predetermined threshold.

(Method of extracting related information data based on similarity)
The related information extraction unit (110) of the present invention can also perform the extraction process as follows.
That is, the CPU (10) includes a related information similarity calculation unit (not shown), and the related information similarity calculation unit performs the similarity calculation processing of articles as described above.
Then, the degree of similarity between the reputation information data collected by the data collection unit (100) and determined as the reputation information and the article extracted from the related information DB on the network or the hard disk is calculated.

  According to the similarity calculated by the related information similarity calculation unit, for example, information whose similarity exceeds a predetermined threshold can be extracted as related information.

(Method of extracting by similarity after paraphrasing)
In the present invention, the CPU (10) is further provided with a word string replacement unit (not shown) having the above paraphrasing technique and the related information similarity calculation unit, and the word string replacement unit extracts from the related information DB or the like. The related information similarity calculation unit calculates the similarity with the reputation information data while sequentially replacing the word strings included in the articles. Then, the word string is replaced so that the degree of similarity always increases, and when the degree of similarity exceeds the predetermined threshold, the information can be extracted as related information data.

In the present invention, the following technique can also be provided for the output processing in the reputation information output unit (103).
One of them is to output a plurality of pieces of reputation information determined by the machine learning determination unit (102) as the reputation information by clustering processing.

(A) Description of clustering There are various methods for clustering. The general ones are described below.

(Description of hierarchical clustering (bottom-up clustering))
Connect the closest members together to create a cluster. Clusters and clusters
No (clusters and members) also connect the nearest clusters.
Since there are various definitions of the distance between clusters, it will be described below.

・ The distance between cluster A and cluster B is the distance between cluster A and cluster B members.
The method of setting the smallest distance among the distances
・ The distance between cluster A and cluster B is the distance between cluster A and cluster B members.
The method of taking the largest distance among the distances
・ Distance between cluster A and cluster B, all members of cluster A and cluster B
The average of the members' distance is the distance
・ The distance between cluster A and cluster B, and the average position of all cluster A members
The cluster position, and the average of all cluster B member positions
And the average of the distances between the positions is the distance.

There is also a method called the Ward method. Hereinafter, the Ward method will be described.
(Equation 16)
W = ΣΣ (x (i, j)-ave _x (i)) ^ 2
^ Means exponent.

First
Σ is an addition from i = 1 to i = g
Second
Σ is an addition from j = 1 to j = ni
x (i, j) is the position of the j-th member of the i-th cluster
ave _x (i) is the average of the positions of all members of the i-th cluster
As the clusters are joined together, the value of W increases, but in the Ward method, the value of W increases.
Connect the clusters together so that they do not become as large as possible.

The position of members is taken from the article, and the type of the word is taken as the vector dimension.
The value of a word vector element can be expressed as the frequency of the word, tf ・ idf of the word (i.e. tf (w, D) * log (N / df (w))), the Okapi expression of the word (i.e. tf Create a vector (w, D) / (tf (w, D) + length / delta) * log (N / df (w))) and use it as the position of the member.

(Description of top-down clustering (non-hierarchical clustering))
Hereinafter, a method of top-down clustering (non-hierarchical clustering) will be described.

(Explanation of maximum distance algorithm)
Take a member. Next, take the member farthest from that member. These members will be the center of each cluster. The minimum distance between each cluster center and the member is taken as the distance of each member, and the member with the largest distance is the center of the new cluster. Repeat this. When the number of clusters reaches a predetermined number, stop repeating. Moreover, the repetition is stopped when the distance between the clusters is equal to or less than a predetermined number. There is also a method to stop the repetition by evaluating the goodness of the cluster based on the AIC information criterion and using the value. Each member becomes the closest cluster-centered member.

(Explanation of K-means method)
Consider clustering into a predetermined number k. Choose k members randomly, and use it as the center of the cluster. Each member becomes the closest cluster-centered member. The average of each member in the cluster is the center of each cluster. Each member becomes the closest cluster-centered member. Moreover, the average of each member in a cluster is made into the center of each cluster. Repeat these. When the center of the cluster stops moving, it stops repeating. Or, repeat it a predetermined number of times. The cluster is obtained using the cluster center at the time of the final cluster center. Each member becomes the closest cluster-centered member.

(Clustering by word group)
As a document classification method similar to clustering, there is a method in which one or a plurality of word groups are defined in advance for each classification destination and classified into classification destinations based on whether or not the input information includes the word group. . The document classification method is also included in the clustering in the present invention.
When the input information includes a plurality of classification target word groups, the input information may be classified into the word group classification destinations with a large number of classifications, and a weight value is assigned to each word group. The words may be classified into the word group having a large weight.

In this way, clustering is performed. Since various methods other than these are known as clustering methods, they may be used.
As for the reputation information, a plurality of similar information is often extracted, and if a plurality of reputation information is output at random, there is a problem that it is difficult to determine which reputation information is really problematic.
In the present invention, clustering processing is performed by the reputation information output unit (103), so that similar reputation information can be displayed on the monitor (12) or the like. For example, even when erroneous information is concentrated, it can be quickly performed. It can be accurately grasped.

In this invention, the display mode at the time of outputting in a reputation information output part (103) can also be changed as follows.
That is, the reputation information output unit (103) counts the number of judgments for each day, weekly, and monthly of the data judged to be reputation information, and creates judgment number data. For example, weekly announcement data as shown in FIG. 10 is created.

 The weekly announcement data shown in FIG. 10 includes, for example, one case in the third week, five cases in the fourth week, ten cases in the sixth week, and seven pieces for the reputation information 1 clustered as described above. There is one judgment per week, and for reputation information 2, there are 5 documents in the 1st week, 3 in the 2nd week, 10 in the 3rd week, and 1 document in the 8th week. There are announcements, and about reputation information 3, 2 cases in the 4th week, 4 cases in the 7th week, 12 cases in the 8th week, 5 cases in the 9th week, 13 cases in the 10th week It indicates that there is a judgment number.

 The reputation information output unit (103) can also be configured to convert the regular announcement data into contour line data and use the converted contour line data as display data. As shown in FIG. 11, the number of presentations can be represented by contour lines, and the color can be displayed darkly according to the height.

  On the monitor (12), the display data created by the reputation information output unit (103) is displayed on the screen. For example, as shown in FIG. 11, the monitor (12) displays a screen on which data of the number of document announcements in each week of each reputation information is displayed in contour lines. The display color of the contour line varies depending on the number of presentations. For example, the display color of contour lines corresponding to the number of presentations of 8 to 10 is displayed in the darkest color.

In the display order of FIG. 11, the average value, the mode value, and the median value for each week are obtained for the number of documents published, and the average values are displayed in ascending order. By arranging in this way, it is possible to display from a category in which announcements are concentrated at an early stage, so it is possible to visually recognize how reputation information and related information were announced.
Note that the arrangement order may be an average value, a mode value, or a median value, and a calculation method using them is also arbitrary.

  In the contour line graph display, a display using a plurality of line graphs or a display using one line graph for each classification may be performed.

For example, as shown in FIG. 12, the monitor (12) can adopt a configuration in which the number of data judgments for each week of each reputation information is displayed on a screen as a bubble chart.
A bubble chart generally refers to a diagram in which (circle) indicating a certain event is arranged on a diagram having two axes. In the bubble chart shown in FIG. 12, the size of the circle indicates the degree of determination.

When outputting reputation information in the present invention, it is also possible to output after summary processing. That is, even if the reputation information is output in a long sentence, it takes time for the user to understand what the content is, and it is inappropriate for checking a large amount of reputation information.
Therefore, in the present invention, the output of reputation information can be presented in an easy-to-understand manner by the following summary process.

First, various known methods are known for the summarization process. For example, the summarization methods disclosed in the methods of Patent Document 2 and Patent Document 3 by the present inventors can be used.
That is, according to the method of Patent Document 2, as summarization apparatus, solution data that stores solution data composed of a combination of a sentence and a problem that is a summary result thereof and a plurality of classification destination solutions that indicate evaluation on the summary result Based on the storage means and the sentences and summary results that are the problem of the solution data, for example, the information indicating the smoothness of the sentences of the summary results and the predetermined information including the information indicating whether or not the summary results display the contents of the sentences Extract as

  A solution feature pair extraction unit that generates a set of the solution and a set of features; a machine learning unit that stores a set of the solution and the set of features as a learning result in a learning result storage unit; and a solution-feature Based on a set of feature extraction means for extracting the set of features from the input text and information extracted by the pair extraction means, and the solution and the set of features as a learning result, And a probability estimation unit that obtains the probability of each classification in the case of the set of text features obtained from the feature extraction unit based on the theorem, and uses the category having the largest probability value as an estimated solution to be obtained.

  The method described in Patent Document 3 is a solution data editing processing device that edits solution data used in processing for automatically summarizing text by a machine learning method, and displays summary results of text on a display device. Means for accepting an evaluation input for the summary result and evaluating the summary result; and a solution data output process for outputting solution data with the text and the summary result as problems and using the evaluation as a solution Means.

Patent No. 3682529 JP2003-248676

All of the above-mentioned methods are intended to improve the accuracy of machine learning by evaluating or feeding back the results of the methods to known summarization methods. Contribute.
Of course, the machine learning method used here is not limited to the simple Bayes method, and any model such as a k-nearest neighbor method, a decision list method, a maximum entropy method, a support vector machine method, or a neural network method may be used.

Note that the summarization processing in the present invention is not limited to the method based on machine learning, and any known summarization method can be used.
For example, using the information on the position of the document, the number of words containing the title, and the tfidf value of the word that appears in the sentence, the score of each sentence is obtained, and the score with the highest score is used as the summary result. There is a way.
In general, since the title word is important, a sentence including many title words can be extracted and used as a summary result.
More simply, the first sentence such as the first sentence of the document may be a summary sentence.

  Furthermore, in the present invention, the data collection unit (100) can also collect foreign language data. Then, the word to be evaluated, the word indicating the content of the reputation, and other features are first machine-translated in the translation unit, and the feature extraction unit (101) extracts the translated features from the collected data. To do.

  As for machine translation, high-accuracy translation may be performed by a known machine translation method, or a foreign language dictionary may be listed on the hard disk and a word may be translated verbatim by simply referring to the foreign language dictionary.

The data targeted by the present invention is not limited to Japanese, and it is possible to extract reputation information of foreign languages by performing all processes such as machine learning on any foreign language.
Further, the foreign language reputation information extracted in this way may be translated into Japanese by inputting it into a known machine translation processing apparatus or machine translation program and output.

According to the present invention, the following reputation information can be determined by the above configuration. The experimental example is shown.
First, prepare the following teacher data.

[Teacher data 1]

Writing: “… bank goes bankrupt”
Sender: Reliable characters: Black background: White patent paper Newspaper basis: None Reputation (correct information)

[Teacher data 2]

Writing: “… bank goes bankrupt”
Sender: Reliable or unknown character: Black background: White patent paper newspaper basis: Yes, not popular (correct information)

[Teacher data 3]

Writing: “… bank goes bankrupt”
Sender: Reliable or unknown character: Red background: Black patent paper newspaper basis: None Reputation (incorrect information)

[Teacher data 4]

Writing: “… the bank is already born”
Sender: Reliable or unknown Character: Black background: White patent paper newspaper basis: None

Extract features from the above teacher data

[Teacher data 1]

Words that were written: “Bank”
Words that were written: “go bankrupt”
Sender: Reliable text: Black background: White patent paper newspaper basis: None

[Teacher data 2]

Words that were written: “Bank”
Words that were written: “go bankrupt”
Sender: Reliable or unknown character: Black background: White patent paper newspaper basis: Yes

[Teacher data 3]

Words that were written: “Bank”
Words that were written: “go bankrupt”
Sender: Reliable or unknown character: Red background: Black patent paper newspaper basis: None

[Teacher data 4]

Words that were written: “Bank”
Words that were written: “I am already born”
Sender: Reliable or unknown characters: Black background: White patent paper newspaper basis: None

It becomes.

The machine learning unit of the machine learning module (FIG. 3) learns from this information what kind of feature is the reputation information and what kind of feature is not the reputation information. as a result,

Sender: Reliable or the basis of a patent paper:

There is a word “bankruptcy”, and if the character: red background: black, the possibility of reputation is high.

After such a learning result is stored in the machine learning result data, the machine learning determination unit performs a determination process. Therefore, it is assumed that the features of the data collected by the data collection unit are as follows.

[Feature of Collected Data 1]

Writing: “… The company goes bankrupt”
Sender: Reliable or unknown character: Red background: Black patent paper newspaper basis: None

At this time,
If there is a word “bankruptcy” and text: red, background: black, it is determined that it is reputation information from the learning result that the reputation is highly likely, and is output.

It is a whole block diagram of the reputation information extraction device of the present invention. It is a process flowchart of the reputation information extraction method of this invention. It is a block diagram of the machine learning module in this invention. It is explanatory drawing explaining the concept of the classification | category of the text in a machine learning process. It is explanatory drawing explaining the concept of the margin by a support vector machine. It is a block diagram of the means to evaluate the data reliability by this invention. 4 is a process flowchart of a method for evaluating data reliability according to the present invention. It is a block diagram of the basis information confirmation means by this invention. It is a processing flowchart of the ground information confirmation method by this invention. It is a figure which shows the output mode (1) from the monitor by this invention. It is a figure which shows the output mode (2) from the monitor by this invention. It is a figure which shows the output mode (3) from the monitor by this invention.

Explanation of symbols

1 Reputation information extraction device 10 CPU
11 Keyboard / Mouse 12 Monitor 13 Hard Disk 14 Network Adapter 100 Data Collection Unit 101 Feature Extraction Unit 102 Machine Learning Determination Unit 103 Reputation Information Output Unit 131 Feature Table 132 Machine Learning Result Data

Claims (8)

A reputation information extraction device using a computer that extracts reputation information published on a network for a predetermined target,
Data collection means for receiving data published from one or more server devices on the network and storing the data as collected data in the collected data storage means;
A feature table storage means for storing a feature table including at least a word or a set of words as a feature;
A feature extraction means for extracting features from the collected data with reference to the feature table;
Machine learning determination means provided with a predetermined machine learning module for determining whether or not it is reputation information for a predetermined object by referring to the machine learning result data stored in the learning result storage means when one or more features are input The feature extraction means inputs the feature extracted to the machine learning judgment means, and obtains a judgment result as to whether or not it is reputation information for the collected data,
Reputation information output means for outputting at least one part of collected data determined as reputation information, or the name or network address of a server device to which it is disclosed, or file information of the collected data
In a configuration comprising:
The meaning of a word is classified as a semantic class using a code, and a classification vocabulary table in which each semantic class is assigned to a plurality of words is included in the feature table as a feature,
The feature extraction means is
A reputation information extracting apparatus , wherein a semantic class of a word included in the collected data is extracted from the collected data with reference to the feature table . In the reputation information extraction device,
For the name or network address of the server device on the network, or the file information of the collected data, an information source reliability database that represents numerically the reliability of information disclosed there is provided, and the reliability is identified As included in the feature table,
The feature extraction means is
The reputation information extraction device according to claim 1, wherein a numerical value of reliability related to the collected data is extracted with reference to the feature table. The reputation information extraction device comprises data reliability value evaluation means,
The data reliability value evaluation means includes:
Evaluation data that matches at least one of the collected data and the author of the collected data, the name or network address of the stored server device, or the file information of the collected data is sent to the server device on the network, or in advance. An evaluation data extraction unit for extracting from the accumulated evaluation database;
An evaluation factor table storage unit that stores at least one of an active factor that is a factor that increases the reliability of the collected data and a negative factor that is a factor that decreases the reliability of the collected data;
An evaluation feature extraction unit that extracts the factor as the feature from the evaluation data;
An evaluation machine provided with a predetermined machine learning module that performs classification according to the reliability of the evaluation data by referring to the machine learning result data stored in the learning result storage unit when one or more features are input And a learning determination unit, and outputs the classification result of the evaluation data obtained by inputting the features extracted by the evaluation feature extraction unit to the evaluation machine learning determination unit as the reliability of the collected data Configuration,
The reliability value of the collection data, together with the feature extracting means has extracted feature, claim input to the machine learning decision means to obtain the reputation information whether the determination result for the collected data 1 Or the reputation information extraction apparatus in any one of 2 . The reputation information extraction device comprises ground information confirmation means,
The basis information confirmation means
A rationale information database that defines a rationale information source of at least one of the name or network address of a server device that publishes reliable information or file information of the reliable information;
A similarity determination unit that acquires data published from a basis information source defined in the basis information database, and determines whether or not similar data similar in topic to the collected data is included,
The determination result in the similarity determination unit is input to the machine learning determination unit together with the feature extracted by the feature extraction unit, and a determination result as to whether or not the collected data has reputation information is obtained. Reputation information extraction apparatus in any one of 3 thru | or 3 . A method for extracting reputation information using a computer that extracts reputation information published on a network for a predetermined target,
A data collection step in which the data collection means of the computer receives data published from one or more server devices on the network and stores them as collected data in the collected data storage means;
A feature table storage means for storing a feature table including at least a word or a set of words as a feature;
A feature extraction step in which a feature extraction unit of the computer refers to the feature table to extract a feature from the collected data;
The machine learning determination means of the computer having the predetermined machine learning module uses the extracted feature and refers to the machine learning result data stored in the learning result storage means to determine whether or not the reputation information is for a predetermined object. A machine learning determination step for determining,
The reputation information output means of the computer outputs at least one of the collected data determined as reputation information, the name or network address of the server device to which it is disclosed, or the file information of the collected data In a configuration including a reputation information output step to
The meaning of a word is classified as a semantic class using a code, and a classification vocabulary table in which each semantic class is assigned to a plurality of words is included in the feature table as a feature,
In the feature extraction step, the feature extraction means includes:
A reputation information extracting method , wherein a semantic class of a word included in the collected data is extracted from the collected data with reference to the feature table . For the name or network address of the server device on the network, or the file information of the collected data, an information source reliability database that represents numerically the reliability of information disclosed there is provided, and the reliability is identified As included in the feature table,
In the feature extraction step, the feature extraction means includes:
The reputation information extraction method according to claim 5 , wherein a numerical value of reliability related to the collected data is extracted with reference to the feature table. A data reliability evaluation step at any point after the data collection step of the reputation information extraction method and before the machine learning determination step;
In the data reliability evaluation step,
The evaluation data extraction unit in the data reliability value evaluation means of the computer has at least one of the collected data, the author of the collected data, the name or network address of the server device stored, or the file information of the collected data Evaluation data extraction processing step for extracting evaluation data with which they match from a server device on the network or an evaluation database stored in advance,
An evaluation factor table storage unit that stores at least one of an active factor that is a factor that increases the reliability of the collected data and a negative factor that is a factor that decreases the reliability of the collected data;
An evaluation feature extraction processing step in which an evaluation feature extraction unit in the data reliability value evaluation means extracts the factor as a feature from the evaluation data;
Machine learning result data stored in the learning result storage unit using the features extracted in the evaluation feature extraction processing step by the evaluation machine learning determination unit having a predetermined machine learning module in the data reliability value evaluation means The evaluation machine learning determination processing step for performing classification according to the reliability of the evaluation data,
Output the classification result of the evaluation data including as the reliability of the collected data,
In the machine learning determination step, the reliability value of the collected data is input to the machine learning determination unit together with the feature extracted by the feature extraction unit, and a determination result as to whether or not the collected data is reputation information is obtained. The method for extracting reputation information according to claim 5 or 6 . In the reputation information extraction method,
Computer clustering means
About at least one of the reputation information data or the related information data,
Including a clustering processing step of clustering the author or content included in any of the data according to a predetermined clustering formula;
In the output step,
8. The reputation information extraction method according to claim 5, wherein at least one of the reputation information data and the related information data in the clustered state is output.

Images