CN103544255B - Text semantic relativity based network public opinion information analysis method - Google Patents

Text semantic relativity based network public opinion information analysis method Download PDF

Info

Publication number
CN103544255B
CN103544255B CN201310482522.5A CN201310482522A CN103544255B CN 103544255 B CN103544255 B CN 103544255B CN 201310482522 A CN201310482522 A CN 201310482522A CN 103544255 B CN103544255 B CN 103544255B
Authority
CN
China
Prior art keywords
text
information
public
similarity
semantic
Prior art date
Application number
CN201310482522.5A
Other languages
Chinese (zh)
Other versions
CN103544255A (en
Inventor
陶宇炜
谢爱娟
熊长江
王娟琳
Original Assignee
常州大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 常州大学 filed Critical 常州大学
Priority to CN201310482522.5A priority Critical patent/CN103544255B/en
Publication of CN103544255A publication Critical patent/CN103544255A/en
Application granted granted Critical
Publication of CN103544255B publication Critical patent/CN103544255B/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/95Retrieval from the web
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00Handling natural language data
    • G06F40/30Semantic analysis

Abstract

The invention relates to a text semantic relativity based network public opinion information analysis system. The system comprises a network public opinion information acquisition module, a public opinion information extraction module, a public opinion information preprocessing module, a public opinion information mining module and a public opinion information analysis module. The network public opinion information acquisition module is used for acquiring various public opinion information rich in content from a webpage. The public opinion information extraction module and the public opinion information preprocessing module are used for preliminarily filtering and segmenting the acquired public opinion information, extracting meta-information of a text part, creating a feature semantic network diagram of texts, and performing weighting computation and feature extraction to provide services for public opinion information mining. The public opinion information mining module is used for classifying the texts by adopting a semantic similarity based improved text clustering analysis method. The public opinion information analysis module is used for performing OLAP (on-line analytical processing) multi-dimensional statistics on mined data of the public opinion information, and analyzing public opinion evaluation indices to provide support for relevant public opinion decision making. By the system, the problem that semantic information of words in the texts is incomplete is solved, and clustering analysis and hot topic extraction of dynamic data in a large-scale network environment are realized efficiently.

Description

The network public sentiment information relevant based on text semantic analyzes method
Technical field
The present invention relates to technical field of network information, a kind of network public sentiment information relevant based on text semantic divides Analysis method.
Background technology
Society, the Internet has penetrated in daily life, the instant messaging work such as microblogging, forum, blog Tool has become as people and obtains information, and then the important channel of the information that gives opinion, propagates.By the network platform, public feelings information Propagating rapidly, cause extensive concern, its speed propagated is soon, scope is extensively, power of influence is big, is far from traditional media comparable, The features such as the anonymous interactivity of cyberspace, non-space-time are restricted, make the public opinion strength that network public-opinion this strand is powerful, to society Development and stabilization can produce certain impact and impact.The network public-opinion in front, like " positive energy ", promotes and promotes social development; Negative network public-opinion forms negative effect to social stability, causes public sentiment crisis.Thus, Strengthens network public feelings information monitoring, Analyzing, manage, to stable society order, building a harmonious society has important practical significance.Prison timely to network public sentiment information Survey, correctly judge decision-making, respond the most in time, actively adopt an effective measure and dissolve public sentiment crisis, become network public-opinion management work The emphasis made and difficulties.
Summary of the invention
Need solve to ask in the feature of network public sentiment information in above-mentioned background technology and network public sentiment information management Topic, the present invention provides a kind of network public sentiment information relevant based on text semantic to analyze method.
The technical solution adopted for the present invention to solve the technical problems is, a kind of network public-opinion relevant based on text semantic Information analysis method.Employing includes that network public sentiment information acquisition module, public feelings information extract module, public feelings information pretreatment mould The network public sentiment information analysis system that block, public feelings information excavate module, public feelings information analyzes module and comprise public feelings information data base System, and comprise the steps:
A. network public sentiment information acquisition module gathers various public feelings information from webpage, and stores public feelings information data base In;
B. the public feelings information that step a is gathered by public feelings information extraction module and public feelings information pretreatment module carries out tentatively mistake Filter and cutting, the content information that extraction text is comprised, excavates for public feelings information and provides data, services;
C., on the basis of step b, public feelings information excavates module and uses improvement Clustering Analysis of Text based on semantic similarity Method, generates classification and describes information, filter out the text message comprised in cluster analysis result;Feature based is utilized to add up TFIDF words-frequency feature computational methods statistics category feature, obtain Based on Class Feature Word Quadric, select noun as candidate categories Feature Words, According to candidate feature word weight sequencing, using the bigger candidate feature word of weighted value as classification key word, utilize classification key word Between semantic relation, formed classification results;Identify and set up new network public-opinion theme, detect, follow the tracks of existing public sentiment theme Related content;
D. last, the data that public feelings information analysis module is excavated public feelings information through step c carry out OLAP multidimensional statistics Analyze, analyze the public sentiment evaluation metrics such as public sentiment subject content attention rate, public sentiment theme Sentiment orientation.
In step a, described public feelings information acquisition module, is to be acquired network public sentiment information source, with general net Unlike network reptile, crawling of its webpage to be completed, and web page contents is formatted process, extract public sentiment Theme and content, the data obtained is stored in txt form or html formatted file, and stores public feelings information data base;Network carriage Feelings information acquisition module uses timesharing to access, IP address is changed in timing and simulation browser carries out three kinds of technology of single-sign-on and combines Carry out anti-shielding.Network public sentiment information acquisition module uses timesharing to access, IP address is changed in timing and simulation browser carries out list Point logs in three kinds of technology combinations and carries out anti-shielding.What network public sentiment information acquisition module performed concretely comprises the following steps: described public sentiment is believed What breath acquisition module performed concretely comprises the following steps, and from the beginning of the URL of predefined theme related web page, obtains the text in webpage Information, and from current web page, extract new URL put in queue, until the public feelings information meeting condition gathers complete, URL team Till being classified as sky;The web page text information collected is stored in public feelings information data base according to field classification, it is provided that public sentiment Information Extracting module is called.
Described public feelings information extraction module, is to remove the irrelevant contents in webpage, as the advertisement in webpage, navigation information, The noise data such as picture, copyright notice, extracts the metamessage of the body part useful to the analysis of public opinion, is reconstructed text, To have the representational information aggregation of theme together;Described public feelings information pretreatment module, is to the public feelings information source gathered After the extraction module extraction of described public feelings information, carry out Chinese word segmentation process, filter stop words, name Entity recognition, part of speech Mark, syntax parsing and Feature Words extract, and set up positive sequence index and inverted index;Set up text feature semantic network figure, with literary composition The entity E comprised in Ben is as the node of figure, and the semantic relation between two entities is as the directed edge of figure, the language between entity Justice relation combines the word frequency information weight as node, and the weight of directed edge represents entity relationship significance level in the text, Described entity E includes things entity NE, event entity VE, event relation entity RE;The word frequency of statistics text and text frequency letter Breath, then carries out Feature Words extraction, and the vocabulary choosing embodiment text feature shows the text.
In stepb, described public feelings information extraction module, it is to remove the irrelevant contents in webpage, extracts the analysis of public opinion The metamessage of useful body part, is reconstructed text, will have the representational information aggregation of theme together;Described carriage Feelings information pre-processing module, be to gather public feelings information source through described public feelings information extraction module extraction after, carry out Chinese Word segmentation processing, filter stop words, name Entity recognition, part-of-speech tagging, syntax parsing and Feature Words extract, set up positive sequence index and Inverted index;Setting up text feature semantic network figure, the entity E comprised in text is as the node of figure, between two entities Semantic relation as the directed edge of figure, the semantic relation between entity combines the word frequency information weight as node, directed edge Weight represent that entity relationship significance level in the text, described entity E include things entity NE, event entity VE, event Relationship entity RE;The word frequency of statistics text and text frequency information, then carry out Feature Words extraction, chooses and embodies text feature Vocabulary shows the text.
The text analyzings such as network public sentiment information text mining to be realized, natural language processing, first have to carry out word segmentation processing, Use for reference the achievement in research in domestic Chinese word segmentation field, use the ICTCLAS Chinese that Inst. of Computing Techn. Academia Sinica develops The functions such as word segmentation that morphological analysis system is had, part-of-speech tagging, name Entity recognition, by public feelings information text Hold and carry out participle, extract the length word more than two.After text participle, filter useless the disabling of computer understanding text Word, retains the word of the parts of speech such as noun, verb, adnoun, dynamic shape word, obtains alternative features word set, effectively reduce the size of index, Increase recall precision, improve accuracy rate.Through the text document of word segmentation processing, set up positive sequence index and inverted index, it is achieved use The inquiry at family is mutual.Text through participle, part-of-speech tagging, remove stop words after, set up the Feature Semantics network of text, statistics literary composition The information such as this word frequency and text frequency, are then weighted and feature extraction etc..
In step c, described public feelings information is excavated module, is text set to be carried out pretreatment, at Chinese word segmentation Reason, stop words filter and after structured tag information analysis, and text data set Information Extracting module generated is special according to text Levy the text semantic feature description structure that semantic network figure builds, utilize method for evaluating similarity to calculate the semantic phase between text Like degree, build similarity matrix, use improvement Clustering Analysis of Text method based on semantic similarity to generate cluster result;Cluster Analysis result generates classification and describes information, filters out the text message comprised in cluster analysis result;Feature based is utilized to add up TFIDF words-frequency feature computational methods statistics category feature, obtain candidate categories Feature Words, select noun special as candidate categories Levy word, according to candidate feature word weight sequencing, determine that candidate feature word, as classification key word, utilizes classification crucial using weighted value Semantic relation between word, forms classification results;Result builds knowledge base, and knowledge base can also be configured with simultaneously Support the text mining function such as public sentiment motif discovery, public sentiment sentiment classification.
In step d, described public feelings information analyze module, be to be stored in public feelings information data base through step c The data excavated carry out OLAP multidimensional statistics analysis, analyze public sentiment theme attention rate, public sentiment content erotic degree, public sentiment Spreading and diffusion Degree, public sentiment issue the public sentiment evaluation metrics such as disturbance degree, grasp in time for relevant departments public sentiment issue dynamically, in good time public feelings information, Make correct decisions and support is provided.
Compared with prior art, the method have the advantages that
1. current network public feelings information has reflected the spies such as magnanimity, dynamic, imperfection, form of expression multiformity Point, and existing public feelings information is analyzed method and is often ignored the dependency relation of public feelings information content of text, causes public feelings information Analysis result is inaccurate;The present invention uses the text feature semantic network graph model building public feelings information text, describes at text Structure introduces the contact between phrase semantic association and context of co-text;In conjunction with improvement text cluster based on semantic similarity Algorithm, mining analysis goes out the content that in public feelings information text, context semanteme is relevant.
2. by setting up the text feature semantic network figure of public feelings information text, by upper between word in public feelings information text Hereafter relation forms characteristic item and the directed graph structure of weight composition, while retaining text word Context information structure, Enhance the intension that in text, word context is semantic, preferably describe semantic information implicit in text and theme feature, solve The certainly problem of phrase semantic loss of learning in text.
3. improvement Text Clustering Algorithm based on semantic similarity is suitable under large-scale network environment dynamic data Cluster analysis and public sentiment theme focus find, by text semantic Similarity Measure, build text semantic similarity matrix, deeply Degree excavates the content that in public feelings information text, context semanteme is relevant, detects in time, follows the tracks of new subject events;In employing class The theme method for expressing at multiple centers, selects similar as this class text of similarity maximum at text center each in class Degree, is effectively improved running efficiency of system, and along with the increase of amount of text, cluster analysis effect can become apparent from.
Accompanying drawing explanation
Fig. 1 is that the embodiment of the present invention analyzes the workflow diagram of method based on the network public sentiment information that text semantic is relevant.
Detailed description of the invention
Below in conjunction with the drawings and specific embodiments, the present invention will be further described.But embodiments of the present invention do not limit In this.
As it is shown in figure 1, in the method for the present invention, including network public sentiment information acquisition module, public feelings information extraction module, carriage Feelings information pre-processing module, public feelings information excavate module, public feelings information analyzes module and the network comprising public feelings information data base Public feelings information analyzes system.Its handling process is:
(1) public feelings information collection
Network public sentiment information source is acquired, unlike general web crawlers, its webpage to be completed Crawl, and web page contents is formatted process, extract useful public feelings information, such as theme and the content of public sentiment, institute Obtain data and be stored in txt form or html formatted file, write original public feelings information data base.Concretely comprise the following steps: according to default Network public sentiment information acquisition strategies, from the beginning of the URL of multiple sub-pages, sends the finger following http agreement by each generic port Make (using GET method);Remote server returns the document of HTML type according to the content of application instruction.Public feelings information gathers mould Block is collected in returning to document and is first preserved to caching after all of information, is then delivered in data base preserve, obtains in webpage Text message;In obtaining web page text information process, from current web page, constantly extract emerging hyperlink URL access, And reject the hyperlink URL accessed, such iterative cycles, until the web page text information gathering meeting search strategy is complete Finish, till the URL queue not accessed is sky.The web page text information gathered is stored in data base according to field classification, carries Call for public feelings information extraction module.
Network public sentiment information acquisition module generally uses timesharing to access, IP address is changed in timing, simulation browser carries out list The anti-shielding strategy that the multiple technologies such as some login combine.For many website such as forums, blog, microblogging etc. by the user side of login Formula could access, and uses the strategy of simulation browser to be easier to realize here, utilizes microemulsion sample injection developing instrument Visual The Web Browser control that Studio2008 provides is the API Calls of MS internet explorer, utilizes the simulation of SSO single-sign-on to carry Handing over user name and password login, after waiting that user login information has loaded, page jump to corresponding URL address, by submitting to Key word is retrieved, it is thus achieved that the source file of required webpage.
The web page text information gathered includes web content information, Web structure and uses record information two parts.Web content Information comprises the content of text information such as headline, body matter, review information, Web structure and Web and uses record information to comprise The statistical information such as click volume, pageview, comment amount.
(2) public feelings information extraction
The info web gathered contains the noise datas such as advertisement, navigation information, picture, copyright notice, divides public feelings information Really it is desirable that the metamessage of body part for analysis, dispose these irrelevant contents, extract and public feelings information is analyzed useful The metamessage of body part, for the follow-up excavation of text, analyze service be provided.Idiographic flow is as follows:
(2-1) align web page text first by Tidy instrument and carry out HTML markup standardization, then utilize html parser Tools build HTML tree, using HTML markup as the node of tree, so represents and is easy to the management to HTML code and operation, permissible Preferably code is carried out structuring excavation.
(2-2) from the public feelings information source gathered, the phases such as title, key word, text, length, renewal time and URL are extracted Pass information, title can intercept label<tITLE>with</TITLE>between information;Key word is included in html file head META label, can extract from META label information;Temporal information can be extracted by pattern match analysis and web page analysis.
(2-3) what text extracted concretely comprises the following steps: select suitable key word, obtains the URL address of related web page, passes through Access the server at place, URL address, obtain the html source code of webpage;Delete the useless labelling row in web page source code, protect Stay webpage body content;By in HTML code paragraph symbol (as</p>,<br>deng) replace with special symbol (such as * [/p] *, * [/br] * etc.), carriage return character and newline replace with line Separator, use row structure storage mode, retain web page contents form;Carry Take the text between every a line HTML markup "<" and ">";Special symbol (such as * [/p] *, * [/br] * etc.) is replaced with the carriage return character, Keep the original paragraph of text;Result character string is removed the special ESC of HTML (such as &quot, &lt etc.) process, knot Close regular expression, mate and extract final text result.
The relevant informations such as title, key word, text, length, renewal time and URL are extracted from the public feelings information source gathered After, the reconstruct of public feelings information extraction module text message to be realized.
Text reconstruct is by analyzing the public feelings information existence form such as Internet news, forum postings, microblogging blog article and text Architectural feature, forms " purport block " by the information of representative topic, information composition " content blocks " of remainder, to improve Cluster analysis effect.
Text for web page news reconstructs, and is the title of web page news and first segment information composition " purport block ", remaining News describe information and comment content composition " content blocks ".
Text for forum postings reconstructs, and is that title and the main note of model are formed " purport block ", by money order receipt to be signed and returned to the sender and follow-up Inforrnation purifying processes, and removes and does not has the model of Chinese character content and use the conventional model evaluating word, selects some models to constitute " content blocks ".
(3) public feelings information pretreatment
Public feelings information extraction after, followed by Chinese word segmentation process, name Entity recognition, part-of-speech tagging, syntax parsing, The pretreatment such as Feature Words extraction, are saved in result in data base.At network public sentiment information text mining to be realized, natural language The text analyzings such as reason, first have to carry out word segmentation processing, use for reference the achievement in research in domestic Chinese word segmentation field, use the Chinese Academy of Sciences Chinese lexical analysis system ICTCLAS of Institute of Computing Technology development carries out participle and the part-of-speech tagging of text, by Chinese Word segmentation processing, extracts the length word more than two.The function of ICTCLAS has the participle of Chinese text, part-of-speech tagging, new word identification Deng;The method using actor model (role model) is named Entity recognition;Support that user defines as required individual simultaneously Property dictionary, not only has the higher precision of word segmentation, and participle effect is preferable.Code is as follows:
After text participle, filter the stop words useless to computer understanding text, retain noun, verb, adnoun, The word of the parts of speech such as dynamic shape word, obtains alternative features word set, to avoid the lengthy and jumbled of text, effectively reduces the size of index, increases inspection Rope efficiency, improves retrieval rate.
Through the text of word segmentation processing, set up positive sequence index and inverted index, it is achieved the inquiry of user is mutual.For positive sequence Index, according to the sequence of word frequency, selects top n word to represent text, is expressed as with Hash table:<filename, key word phrase>; After setting up positive sequence index, the key word in search text, find out the All Files name comprising this key word, set up file noun Group, can obtain inverted index, be expressed as with Hash table:<key word, filename phrase>.
The foundation of index and the retrieval service of index realize based on Apache open source projects Lucene, and Lucene provides complete Query engine and index engine, text analyzing engine;Use Hadoop storage and the index file of management magnanimity.
Index to set up process as follows:
1. create index and write object IndexWriter.Vocabulary resolver, different vocabulary solutions need to be provided during this Object Creation Parser uses different dictionaries.Select ThesaurusAnalyzer, it is possible to extract synopsis;
2. for taking from each result set one the Document object of establishment in data base;
3. the data element in result set is respectively created a Field object, and adds Document object to;
4. write this Document object.
The process of indexed search is: first create query parser, and this query parser needs Field object name and right The parameters such as the vocabulary resolver answered;Query object is obtained again by query parser and keyword;Retrieval is obtained by query object Result set, result set is made up of Document object.
Text through participle, part-of-speech tagging, remove stop words after, set up the Feature Semantics network of text, statistics text The information such as word frequency and text frequency, are then weighted and feature extraction etc..
Text feature semantic network figure be a kind of entity and semantic relation thereof to express the directed graph of public feelings information, with literary composition The entity E(comprised in Ben includes things entity NE, event entity VE, event relation entity RE) as the node of figure, two realities Semantic relation between body is as the directed edge of figure, and the semantic relation between entity combines the word frequency information weight as node, The weight of directed edge represents entity relationship significance level in the text.By the introducing of network node weights and based on concept Merging and simplify, building text feature semantic network figure, the core extracting text is semantic.The word i.e. represented by network node Merging, node weights are added;Remerging directed edge, directed edge weights are added, and build text feature semantic network figure, describe text In semantic information and theme feature.Concrete concept is described as follows:
C1: things entity NE is defined as NE(id, concept, property, power).Id represents entity identification, Concept represents entitative concept, and property represents entity attribute, and power represents weight.
C2: event entity VE is defined as VE(id, concept, property, power, isN, subT, objT1, ObjT2).In addition to the several data item comprising NE, whether isN represents is negative, and subT represents main body entity gauge outfit, objTl With the gauge outfit that objT2 represents object entity 1 and 2.
C3: event relation entity RE is defined as RE(id, concept, property, power, isN, subT, objT).RE Just can be fully described with a pair Subjective and Objective entity.
Text feature semantic network graph model analytical procedure is as follows:
S1: when analyzing text, first in units of statement, build each bar statement characteristic of correspondence semantic network figure.By Sentence is analyzed every and is created which NE, and NE and attribute information thereof are charged to entity information table.
After S2:NE analyzes, analyze VE, the concept of registration VE, attribute, subject and object.The VE that Subjective and Objective is identical is real Body surface is shown as same VE, otherwise arranges different id.
S3: next analyze RE.RE is it is noted that make a distinction with NE, VE, the concept of RE, attribute, main body, object in analysis It is registered in entity information table.
S4: analyze after terminating, obtain the entity information table of this statement.Entity information table describes the relation between entity, It is used for constructing entity relationship diagram, between NE and VE, between RE and NE, VE, by different line handles between entity E from attribute T Entity relationship visualizes.
S5: on the basis of analyzing the Feature Semantics network building first statement, by the Feature Semantics net of follow-up statement Network figure merges, and first merges node, remerges directed edge.
S6: when merging node, the node identical for word between node or semantic similarity being met threshold condition merges, Node weights are added;Otherwise retain this node.
S7: directed edge merge, be merge after node between exist directed edge merge, directed edge weights be added.
S8: update the new weights that weights are this node merging node adjacency limit, the semantic relation between strengthening node.
S9: after exporting the Feature Semantics network of all merging statements, completes the Feature Semantics network of whole text Structure.
Next step is to part of speech feature weight assignment, accurately to indicate text.Retouch according to Chinese part of speech feature and complete event State key element (time, place, personage and event content), in conjunction with Chinese Academy of Sciences's Chinese part of speech label sets, text feature weight Assignment is divided into: title weighted value is 3, and subtitle and keyword weight value are 2, and summary weighted value is 1.5, the first sentence of section and section tail sentence Weighted value is 1.3.
Public feelings information is after pretreatment, and title, text and reply for text arrange different labels, is calculating weight Time, read the label information of key word, complete the assignment of the position weight of word.
(4) public feelings information excavates
Public feelings information excavate module, be that text set is being carried out pretreatment, including Chinese word segmentation process, stop words filter and After structured tag information analysis, text data set Information Extracting module generated, according to text feature semantic network figure structure The text semantic feature description structure built, utilizes method for evaluating similarity to calculate the semantic similarity between text, builds similar Degree matrix, uses improvement Clustering Analysis of Text method based on semantic similarity to generate cluster result;Cluster analysis result generates Classification describes information, filters out the text message comprised in cluster analysis result;The TFIDF word frequency utilizing feature based to add up is special Levying computational methods statistics category feature, obtain candidate categories Feature Words, selection noun is as candidate categories Feature Words, according to candidate Term weight function sorts, and determines that candidate feature word, as classification key word, utilizes the semanteme between classification key word using weighted value Relation, forms classification results;Result builds knowledge base, and knowledge base can also be configured with support public sentiment theme simultaneously The text mining functions such as discovery, public sentiment sentiment classification.
First between the similarity defined and calculate between text, i.e. text, the degree of correlation of discussed theme, uses Sim (D1,D2) represent text D1With text D2Between similarity.Similarity span between zero and one, with text D1And D2Phase It is directly proportional like degree.Similarity between text is the biggest, shows that the theme correlation degree between text is the biggest.Language between text Justice method for evaluating similarity is as follows:
If the public feelings information through step b extracts and pretreated text is D1(t11,t12,t13,…,t1m), D2(t21, t22,t23,…,t2m), calculate text D1In all key word t1iWith text D2In all key word t2iSimilarity, formed similar Degree matrix is as follows:
M ( D 1 , D 2 ) = Sim 11 Sim 1 m Sim m 1 Sim mm
Simij(1=i, j=m) represents text D1Key word t1iWith text D2Key word t2jSimilarity;M(D1,D2) represent Text D1With text D2Between similarity matrix;I is text D1Key word number;M is text D2Key word number;
Word similarity formula is: S (T1,T2)=Max(i=1,2,…,n;j=1,2,…,m)S(y1i,y2j), i.e. word Language similarity is the maximum in the two all senses of a dictionary entry of word (multiple meaning of a word that a word is comprised) similarity.
Traversal similarity matrix M successively, finds the key word correspondence combination that similarity Sim value is maximum, and deletes correspondence Row and column.Then proceeding to travel through similarity matrix M and find the maximum key word combination of Similarity value, iterative cycles is until matrix M For null value matrix.Finally utilize the similarity maximum key word composite sequence obtained, try to achieve text D1And D2Semantic similarity, Computing formula is as follows:
Sim ( D 1 , D 2 ) = 1 m &Sigma; k = 1 m Sim k - max ( t 1 i , t 2 j )
Wherein, max is the maximum of similarity Sim;I is text D1Key word number;J is text D2Key word number.
Improvement Clustering Analysis of Text method based on semantic similarity, is described as follows:
First, to the text of all collections after pretreatment, use TFIDF weighting method that all categories key word is entered Row characteristic weighing, extracts m optimal characteristics key word and is formed original based on keyword feature vector Di*.
2. according to described knowledge base, original is carried out pretreatment based on key word in keyword feature vector Di*: knowing Know and the vocabulary with Keywords matching is found in storehouse and is replaced, form new characteristic vector Di, Di=(T1,T2,…,Ti),i=1, 2,3,…,m。
3. form m characteristic vector D of n texti, utilize text semantic calculating formula of similarity to calculate the text gathered Between semantic similarity, form the similarity matrix M of text set, and obtain the average similarity MA of all characteristic vectors.Meter Calculation formula is as follows:
M = S 11 S 1 n S n 1 S nn , MA = &Sigma; i = 1 n &Sigma; j = 1 n Sij - n &Sigma; i = 1 n Sii n * ( n - 1 ) ; Wherein, n is textual data;
4. setting three similarity thresholds, a multiplicity threshold value is 0.9, and a theme central threshold is 0.5, Yi Jiyi Individual new theme threshold value is 0.3;
5. text is compared with central theme, if the initial center similarity of text and central theme is more than multiplicity threshold Value 0.9, it is believed that the text belongs to the same content text of same subject;If similarity is less than new theme threshold value 0.3, the then text Need a newly-built class;If similarity is in the range of 0~0.5, then the text belongs to the not ipsilateral discussion of same subject Core content text, is labeled as second center, by that analogy, forming the cluster result of the stratification at multiple center.
6., for the theme method for expressing at multiple centers, select text to make with the maximum of the similarity at each center in class Similarity for this class text.
Improvement Text Clustering Algorithm based on semantic similarity is suitable under large-scale network environment gathering dynamic data Alanysis and public sentiment theme focus find, new events can be detected in time, detect, follow the tracks of new public sentiment theme;Use in class many The public sentiment theme method for expressing at individual center, is effectively improved running efficiency of system, and along with the increase of amount of text, effect can be more Add substantially.
5) public feelings information analysis
Described public feelings information is analyzed module and is carried out the data through the excavation of step c being stored in public feelings information data base OLAP multidimensional statistics is analyzed, and analyzes the public sentiment evaluation metrics such as public sentiment subject content attention rate, public sentiment theme Sentiment orientation, is relevant Department grasps public sentiment in time and issues public feelings information dynamically, in good time, makes correct decisions offer support.
By the public sentiment theme gathering, processing and mining analysis produces, it is expressed as: T=(T1,T2,…,Tn), wherein TiTable Show the text of public sentiment theme.The attention rate of public sentiment subject text is expressed as: Ti=αNp+βNr, the attention rate tolerance public affairs of public sentiment theme Formula is:Wherein α, β represent weight, NpRepresent the hits of public sentiment subject text, NrRepresent comment number;Np_i represents the hits of i-th public sentiment subject text, and Nr_i represents commenting of i-th public sentiment subject text Opinion number.Due to Np>Nr, through statistics, α value is 0.02, and β value is 0.98.
The Sentiment orientation of public sentiment theme cluster analysis based on public sentiment subject text data describe.First a fault is set Value, only when the tendency metric of text is more than threshold, text just shows polarity (front property, negative).The tendency of text Metric is just, then the text is the comment in front, otherwise is then negative comment.
Public feelings information passes through collection, pretreatment, Information Extracting, excavates and analyze, and can obtain the detailed number of public sentiment theme According to, processing according to the public sentiment indicator evaluation system set up, the result of process provides decision-making to help.

Claims (7)

1. analyze method based on the network public sentiment information that text semantic is relevant, it is characterised in that: use and include network public sentiment information Acquisition module, public feelings information extraction module, public feelings information pretreatment module, public feelings information excavate module, public feelings information analyzes mould Block and the network public sentiment information comprising public feelings information data base analyze system, and comprise the steps:
A. network public sentiment information acquisition module gathers various public feelings information from webpage, and stores in public feelings information data base;
B. public feelings information extraction module and public feelings information pretreatment module public feelings information that step a is gathered tentatively filter with Cutting, the content information that extraction text is comprised, excavates for public feelings information and provides data, services;
C., on the basis of step b, public feelings information excavates module and uses improvement Clustering Analysis of Text method based on semantic similarity, Generate classification and describe information, filter out the text message comprised in cluster analysis result;Utilize the TFIDF word that feature based is added up Frequently feature calculation method statistic category feature, obtains Based on Class Feature Word Quadric, and selection noun is as candidate categories Feature Words, according to candidate Term weight function sorts, and using the bigger candidate feature word of weighted value as classification key word, utilizes the language between classification key word Justice relation, forms classification results;Identify and set up new network public-opinion theme, detect, follow the tracks of inside the Pass the phase of existing public sentiment theme Hold;
D. last, public feelings information is analyzed module and public feelings information is carried out OLAP multidimensional statistics analysis through the data that step c is excavated, Analyze the public sentiment evaluation metrics such as public sentiment subject content attention rate, public sentiment theme Sentiment orientation;
In step a, described public feelings information acquisition module, is to be acquired network public sentiment information source, webpage to be completed Crawl, and web page contents is formatted process, extract theme and the content of public sentiment, the data obtained is stored in txt lattice Formula or html formatted file, and store public feelings information data base;Network public sentiment information acquisition module uses timesharing to access, regularly Change IP address and simulation browser carries out three kinds of technology combinations of single-sign-on and carries out anti-shielding.
The network public sentiment information relevant based on text semantic the most according to claim 1 analyzes method, it is characterized in that, described What public feelings information acquisition module performed concretely comprises the following steps, and from the beginning of the URL of predefined theme related web page, obtains in webpage Text message, and from current web page, extract new URL put in queue, until the public feelings information meeting condition has gathered Finish, till URL queue is sky;The web page text information collected is stored in public feelings information data base according to field classification, Public feelings information extraction module is provided to call.
The network public sentiment information relevant based on text semantic the most according to claim 1 analyzes method, it is characterized in that, in step In rapid b, described public feelings information extraction module, it is to remove the irrelevant contents in webpage, extracts the textual useful to the analysis of public opinion The metamessage divided, is reconstructed text, will have the representational information aggregation of theme together;Described public feelings information pretreatment Module, be to gather public feelings information source through described public feelings information extraction module extraction after, carry out Chinese word segmentation process, filtration Stop words, name Entity recognition, part-of-speech tagging, syntax parsing and Feature Words extract, and set up positive sequence index and inverted index;Set up Text feature semantic network figure, the entity E comprised in text is as the node of figure, the semantic relation conduct between two entities The directed edge of figure, the semantic relation between entity combines the word frequency information weight as node, the weight presentation-entity of directed edge Relation significance level in the text, described entity E includes things entity NE, event entity VE, event relation entity RE;Statistics The word frequency of text and text frequency information, then carry out Feature Words extraction, and the vocabulary choosing embodiment text feature shows the text.
The network public sentiment information relevant based on text semantic the most according to claim 3 analyzes method, it is characterized in that, in step In rapid c, described public feelings information excavates module, is that text set is being carried out pretreatment, filters including Chinese word segmentation process, stop words After structured tag information analysis, text data set Information Extracting module generated, according to text feature semantic network figure The text semantic feature description structure built, utilizes method for evaluating similarity to calculate the semantic similarity between text, builds phase Seemingly spend matrix, use improvement Clustering Analysis of Text method based on semantic similarity to generate cluster result;Cluster analysis result is raw Become classification to describe information, filter out the text message comprised in cluster analysis result;Utilize the TFIDF word frequency that feature based is added up Feature calculation method statistic category feature, obtains candidate categories Feature Words, and selection noun is as candidate categories Feature Words, according to time Select term weight function to sort, determine that candidate feature word, as classification key word, utilizes the language between classification key word using weighted value Justice relation, forms classification results;Result is built knowledge base.
5. analyze method according to the network public sentiment information relevant based on text semantic described in claim 3 or 4, it is characterized in that, Text feature semantic network figure is the directed graph utilizing entity and semantic relation thereof to express public feelings information, by network node table The word shown merges, and node weights are added;Remerging directed edge, directed edge weights are added, and build text feature semantic network figure, Semantic information in text and theme feature are described.
The network public sentiment information relevant based on text semantic the most according to claim 4 analyzes method, it is characterized in that, text Between semantic similarity evaluation methodology be:
If the public feelings information through step b extracts and pretreated text is D1(t11,t12,t13,…,t1m), D2(t21,t22, t23,…,t2m), calculate text D1In all key word t1iWith text D2In all key word t2iSimilarity, formed similarity Matrix is as follows:
M ( D 1 , D 2 ) = Sim 11 Sim 1 m Sim m 1 Sim m m
Simij(1=i, j=m) represents text D1Key word t1iWith text D2Key word t2jSimilarity;M(D1,D2) represent text D1With text D2Between similarity matrix;I is text D1Key word number;M is text D2Key word number;
Word similarity formula S (T1,T2)=Max(i=1,2 ..., n;J=1,2 ..., m)S(y1i,y2j), i.e. Words similarity is two words Maximum in language all senses of a dictionary entry similarity, the described senses of a dictionary entry refers to multiple meaning of a word that a word is comprised;
Traversal similarity matrix M successively, finds the key word correspondence combination that similarity Sim value is maximum, and delete the row of correspondence with Row;Then proceeding to travel through similarity matrix M and find the key word combination of Sim value maximum, iterative cycles is until matrix M is null value square Battle array;Finally utilize the similarity maximum key word composite sequence obtained, try to achieve text D1And D2Semantic similarity, computing formula As follows:
S i m ( D 1 , D 2 ) = 1 m &Sigma; k = 1 m Sim k - m a x ( t 1 i , t 2 j )
Wherein, max is the maximum of similarity Sim;I is text D1Key word number;J is text D2Key word number.
The network public sentiment information relevant based on text semantic the most according to claim 6 analyzes method, it is characterized in that, based on The improvement Clustering Analysis of Text method of semantic similarity is:
1) first to the text of all collections after pretreatment, use TFIDF weighting method that all categories key word is carried out spy Levy weighting, extract m optimal characteristics key word and formed original based on keyword feature vector Di*;
2) according to described knowledge base, original is carried out pretreatment based on key word in keyword feature vector Di*: in knowledge base In find the vocabulary with Keywords matching and replaced, form new characteristic vector Di, Di=(T1,T2,…,Ti), i=1,2, 3,…,m;
3) m characteristic vector D of n text is formedi, utilize text semantic calculating formula of similarity to calculate between the text gathered Semantic similarity, form the similarity matrix M of text set, and obtain the average similarity MA of all characteristic vectors;Calculate public affairs Formula is as follows:
Wherein, n is textual data;
4) setting three similarity thresholds, a multiplicity threshold value is 0.9, and a theme central threshold is 0.5, and one new Theme threshold value is 0.3;
5) text is compared with central theme, if the initial center similarity of text and central theme is more than multiplicity threshold value 0.9, it is believed that the text belongs to the same content text of same subject;If similarity is less than new theme threshold value 0.3, then the text needs Want a newly-built class;If similarity is in the range of 0~0.5, then the text belongs to the core that the not ipsilateral of same subject is discussed Heart content text, is labeled as second center, by that analogy, forming the cluster result of the stratification at multiple center;
6) for the theme method for expressing at multiple centers, select text and the maximum of the similarity at each center in class as this The similarity of class text.
CN201310482522.5A 2013-10-15 2013-10-15 Text semantic relativity based network public opinion information analysis method CN103544255B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201310482522.5A CN103544255B (en) 2013-10-15 2013-10-15 Text semantic relativity based network public opinion information analysis method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201310482522.5A CN103544255B (en) 2013-10-15 2013-10-15 Text semantic relativity based network public opinion information analysis method

Publications (2)

Publication Number Publication Date
CN103544255A CN103544255A (en) 2014-01-29
CN103544255B true CN103544255B (en) 2017-01-11

Family

ID=49967707

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201310482522.5A CN103544255B (en) 2013-10-15 2013-10-15 Text semantic relativity based network public opinion information analysis method

Country Status (1)

Country Link
CN (1) CN103544255B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109446409A (en) * 2018-09-19 2019-03-08 杭州安恒信息技术股份有限公司 A kind of recognition methods of the target object of doubtful multiple level marketing behavior

Families Citing this family (70)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103902659B (en) * 2014-03-04 2017-06-27 深圳市至高通信技术发展有限公司 A kind of the analysis of public opinion method and corresponding device
CN103886051A (en) * 2014-03-13 2014-06-25 电子科技大学 Comment analysis method based on entities and features
CN103927545B (en) * 2014-03-14 2017-10-17 小米科技有限责任公司 Clustering method and relevant apparatus
CN104915359B (en) * 2014-03-14 2019-05-28 华为技术有限公司 Theme label recommended method and device
CN103902674B (en) * 2014-03-19 2017-10-27 百度在线网络技术(北京)有限公司 The acquisition method and device of the comment data of particular topic
CN103838886A (en) * 2014-03-31 2014-06-04 辽宁四维科技发展有限公司 Text content classification method based on representative word knowledge base
CN103841216A (en) * 2014-04-01 2014-06-04 深圳市科盾科技有限公司 Network public opinion monitoring system based on cloud platform
CN104199829B (en) * 2014-07-25 2017-07-04 中国科学院自动化研究所 Affection data sorting technique and system
CN104346425B (en) * 2014-07-28 2017-10-31 中国科学院计算技术研究所 A kind of method and system of the internet public feelings index system of stratification
CN104217718B (en) * 2014-09-03 2017-05-17 陈飞 Method and system for voice recognition based on environmental parameter and group trend data
JP6605022B2 (en) * 2014-09-03 2019-11-13 ザ ダン アンド ブラッドストリート コーポレーションThe Dun And Bradstreet Corporation Systems and processes for analyzing, selecting, and capturing sources of unstructured data by experience attributes
CN104268194A (en) * 2014-09-19 2015-01-07 国家电网公司 Method for dynamically generating public opinion brief report
CN105574047A (en) * 2014-10-17 2016-05-11 任子行网络技术股份有限公司 Website main page feature analysis based Chinese website sorting method and system
CN104504150B (en) * 2015-01-09 2017-09-29 成都布林特信息技术有限公司 News public sentiment monitoring system
CN105992194B (en) * 2015-01-30 2019-10-29 阿里巴巴集团控股有限公司 The acquisition methods and device of network data content
CN104699763B (en) * 2015-02-11 2017-10-17 中国科学院新疆理化技术研究所 The text similarity gauging system of multiple features fusion
CN106156041B (en) * 2015-03-26 2019-05-28 科大讯飞股份有限公司 Hot information finds method and system
CN106156192A (en) * 2015-04-21 2016-11-23 北大方正集团有限公司 Public sentiment data clustering method and public sentiment data clustering system
CN104820629B (en) * 2015-05-14 2018-01-30 中国电子科技集团公司第五十四研究所 A kind of intelligent public sentiment accident emergent treatment system and method
CN106294358A (en) * 2015-05-14 2017-01-04 北京大学 The search method of a kind of information and system
CN104915453A (en) * 2015-07-01 2015-09-16 北京奇虎科技有限公司 Method, device and system for classifying POI information
CN104899339A (en) * 2015-07-01 2015-09-09 北京奇虎科技有限公司 Method and system for classifying POI (Point of Interest) information
CN105183803A (en) * 2015-08-25 2015-12-23 天津大学 Personalized search method and search apparatus thereof in social network platform
CN105183478B (en) * 2015-09-11 2018-11-23 中山大学 A kind of webpage reconstructing method and its device based on color transfer
CN106528581B (en) * 2015-09-15 2019-05-07 阿里巴巴集团控股有限公司 Method for text detection and device
CN106649367B (en) * 2015-10-30 2020-03-03 北京国双科技有限公司 Method and device for detecting keyword popularization degree
CN105279277A (en) * 2015-11-12 2016-01-27 百度在线网络技术(北京)有限公司 Knowledge data processing method and device
CN105389389B (en) * 2015-12-10 2018-09-25 安徽博约信息科技股份有限公司 A kind of network public-opinion propagation situation medium control analysis method
CN105677802A (en) * 2015-12-31 2016-06-15 宁波公众信息产业有限公司 Internet information analysis system
CN105447202A (en) * 2015-12-31 2016-03-30 宁波公众信息产业有限公司 Internet information collecting system
CN105677873B (en) * 2016-01-11 2019-03-26 中国电子科技集团公司第十研究所 Text Intelligence association cluster based on model of the domain knowledge collects processing method
CN105740238B (en) * 2016-03-04 2019-02-01 北京理工大学 A kind of event relation intensity map construction method merging sentence justice information
CN105956070A (en) * 2016-04-28 2016-09-21 优品财富管理有限公司 Method and system for integrating repetitive records
CN105956069A (en) * 2016-04-28 2016-09-21 优品财富管理有限公司 Network information collection and analysis method and network information collection and analysis system
CN106126558B (en) * 2016-06-16 2019-09-20 东软集团股份有限公司 A kind of public sentiment monitoring method and device
CN106294542B (en) * 2016-07-25 2018-03-30 北京市信访矛盾分析研究中心 A kind of letters and calls data mining methods of marking and system
CN106294619A (en) * 2016-08-01 2017-01-04 上海交通大学 Public sentiment intelligent supervision method
CN106776724B (en) * 2016-11-16 2020-09-08 福建天泉教育科技有限公司 Question classification method and system
CN106651696B (en) * 2016-11-16 2020-10-27 福建天泉教育科技有限公司 Approximate question pushing method and system
CN106599054B (en) * 2016-11-16 2019-12-24 福建天泉教育科技有限公司 Method and system for classifying and pushing questions
CN107045524B (en) * 2016-12-30 2019-12-27 中央民族大学 Method and system for classifying network text public sentiments
CN107016068A (en) * 2017-03-21 2017-08-04 深圳前海乘方互联网金融服务有限公司 Knowledge mapping construction method and device
CN107145516A (en) * 2017-04-07 2017-09-08 北京捷通华声科技股份有限公司 A kind of Text Clustering Method and system
CN107066585B (en) * 2017-04-17 2019-10-01 济南大学 A kind of probability topic calculates and matched public sentiment monitoring method and system
CN107085608A (en) * 2017-04-21 2017-08-22 上海喆之信息科技有限公司 A kind of effective network hotspot monitoring system
CN107093021A (en) * 2017-04-21 2017-08-25 深圳市创艺工业技术有限公司 Electricity power engineering goods and materials contract is honoured an agreement sincere public sentiment monitoring system
CN107038156A (en) * 2017-04-28 2017-08-11 北京清博大数据科技有限公司 A kind of hot spot of public opinions Forecasting Methodology based on big data
CN107291808A (en) * 2017-05-16 2017-10-24 南京邮电大学 It is a kind of that big data sorting technique is manufactured based on semantic cloud
CN107220236A (en) * 2017-05-23 2017-09-29 武汉朱雀闻天科技有限公司 It is a kind of to determine the doubtful naked method and device for borrowing student
CN107315778A (en) * 2017-05-31 2017-11-03 温州市鹿城区中津先进科技研究院 A kind of natural language the analysis of public opinion method based on big data sentiment analysis
CN107292743A (en) * 2017-06-07 2017-10-24 前海梧桐(深圳)数据有限公司 The intelligent decision making method and its system invested and financed for enterprise
CN107231570A (en) * 2017-06-13 2017-10-03 中国传媒大学 News data content characteristic obtains system and application system
CN107527289A (en) * 2017-08-25 2017-12-29 百度在线网络技术(北京)有限公司 A kind of investment combination industry distribution method, apparatus, server and storage medium
CN107491438A (en) * 2017-08-25 2017-12-19 前海梧桐(深圳)数据有限公司 Business decision elements recognition method and its system based on natural language
CN107679084A (en) * 2017-08-31 2018-02-09 平安科技(深圳)有限公司 Cluster labels generation method, electronic equipment and computer-readable recording medium
CN107679977A (en) * 2017-09-06 2018-02-09 广东中标数据科技股份有限公司 A kind of tax administration platform and implementation method based on semantic analysis
CN107918644B (en) * 2017-10-31 2020-12-08 北京锐思爱特咨询股份有限公司 News topic analysis method and implementation system in reputation management framework
CN107908694A (en) * 2017-11-01 2018-04-13 平安科技(深圳)有限公司 Public sentiment clustering method, application server and the computer-readable recording medium of internet news
CN108052527A (en) * 2017-11-08 2018-05-18 中国传媒大学 Method is recommended in film bridge piecewise analysis based on label system
CN108170666A (en) * 2017-11-29 2018-06-15 同济大学 A kind of improved method based on TF-IDF keyword extractions
CN108197638B (en) * 2017-12-12 2020-03-20 阿里巴巴集团控股有限公司 Method and device for classifying sample to be evaluated
CN108550380A (en) * 2018-04-12 2018-09-18 北京深度智耀科技有限公司 A kind of drug safety information monitoring method and device based on public network
CN109145085B (en) * 2018-07-18 2020-11-27 北京市农林科学院 Semantic similarity calculation method and system
CN109977995A (en) * 2019-02-11 2019-07-05 平安科技(深圳)有限公司 Text template recognition methods, device and computer readable storage medium
CN110188196A (en) * 2019-04-29 2019-08-30 同济大学 A kind of text increment dimension reduction method based on random forest
CN110119416A (en) * 2019-05-16 2019-08-13 重庆八戒传媒有限公司 A kind of service data analysis system and method
CN110188168A (en) * 2019-05-24 2019-08-30 北京邮电大学 Semantic relation recognition methods and device
CN110348539A (en) * 2019-07-19 2019-10-18 知者信息技术服务成都有限公司 Short text correlation method of discrimination
CN111797333A (en) * 2020-06-04 2020-10-20 南京擎盾信息科技有限公司 Public opinion spreading task display method and device
CN111831922B (en) * 2020-07-14 2021-02-05 深圳市众创达企业咨询策划有限公司 Recommendation system and method based on internet information

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101529418A (en) * 2006-01-19 2009-09-09 维里德克斯有限责任公司 Systems and methods for acquiring analyzing mining data and information
CN101788988A (en) * 2009-01-22 2010-07-28 蔡亮华 Information extraction method
CN102708096A (en) * 2012-05-29 2012-10-03 代松 Network intelligence public sentiment monitoring system based on semantics and work method thereof

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8874581B2 (en) * 2010-07-29 2014-10-28 Microsoft Corporation Employing topic models for semantic class mining

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101529418A (en) * 2006-01-19 2009-09-09 维里德克斯有限责任公司 Systems and methods for acquiring analyzing mining data and information
CN101788988A (en) * 2009-01-22 2010-07-28 蔡亮华 Information extraction method
CN102708096A (en) * 2012-05-29 2012-10-03 代松 Network intelligence public sentiment monitoring system based on semantics and work method thereof

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
基于语义相似度的文本聚类算法的研究;孙爽;《中国优秀硕士学位论文全文数据库 信息科技辑》;20080115(第01期);I140-15 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109446409A (en) * 2018-09-19 2019-03-08 杭州安恒信息技术股份有限公司 A kind of recognition methods of the target object of doubtful multiple level marketing behavior

Also Published As

Publication number Publication date
CN103544255A (en) 2014-01-29

Similar Documents

Publication Publication Date Title
CN106484664B (en) Similarity calculating method between a kind of short text
CN104765769B (en) The short text query expansion and search method of a kind of word-based vector
US10867256B2 (en) Method and system to provide related data
Gokulakrishnan et al. Opinion mining and sentiment analysis on a twitter data stream
Shen et al. Linden: linking named entities with knowledge base via semantic knowledge
WO2016112679A1 (en) Method, system and storage medium for realizing intelligent answering of questions
CN104391942B (en) Short essay eigen extended method based on semantic collection of illustrative plates
CN103838833B (en) Text retrieval system based on correlation word semantic analysis
Rousseau et al. Main core retention on graph-of-words for single-document keyword extraction
Li et al. Twiner: named entity recognition in targeted twitter stream
US7657507B2 (en) Pseudo-anchor text extraction for vertical search
JP6007088B2 (en) Question answering program, server and method using a large amount of comment text
TWI424325B (en) Systems and methods for organizing collective social intelligence information using an organic object data model
CN103324665B (en) Hot spot information extraction method and device based on micro-blog
Rizzo et al. NERD meets NIF: Lifting NLP extraction results to the linked data cloud
CN101246499B (en) Network information search method and system
Ryu et al. Open domain question answering using Wikipedia-based knowledge model
US8185530B2 (en) Method and system for web document clustering
US8285713B2 (en) Image search using face detection
CN102298635B (en) Method and system for fusing event information
CN102929873B (en) Method and device for extracting searching value terms based on context search
CN103064956B (en) For searching for the method for digital content, calculating system and computer-readable medium
Cai et al. Extracting content structure for web pages based on visual representation
CN103678576B (en) The text retrieval system analyzed based on dynamic semantics
CN102054015B (en) System and method of organizing community intelligent information by using organic matter data model

Legal Events

Date Code Title Description
PB01 Publication
C06 Publication
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant