CN103729388A - Real-time hot spot detection method used for published status of network users - Google Patents

Abstract

The invention relates to a real-time hot spot detection method used for the published status of network users. The method includes the steps that hot words are detected according to the frequency of occurrence of words in the status published by the users, and a word is recognized as a hot word when the frequency of occurrence of the word changes in a hopped mode in unit time; a plurality key words most relevant to the hot words are calculated; the multiple key words converge into hot pots. By the application of the real-time hot spot detection method used for the published status of the network users, the complexity of an algorithm can be reduced, the practicability can be enhanced, and a large amount of data can be calculated in real time; according to the characteristics in the social network, the detected hot pots can be automatically classified through some available exclusive characteristics of the social network, and a high accuracy rate and good effects are achieved.

Description

For the network user, deliver the real-time hot spot detecting method of state

Technical field

The present invention relates to social networks application, more specifically, relate to the real-time hot spot detecting method of delivering state for the network user.

Background technology

In the past few years, this concept of social networks is gradually for people is familiar with, social network-i i-platform scale develops rapidly, and its service function providing is more and more abundanter, and focus incident and much-talked-about topic are found and recommend to have become the total major function of nearly all social network-i i-platform.The strategy that at present industry member is found focus is comparatively coarse, and to meet in the various demand of user effect general.

" research of micro-blog much-talked-about topic discovery strategy " (Yang Guanchao, Zhejiang University) from current academia, the method for some phenomenal researchs microblogging platform and solution much-talked-about topic discovery is at present started with, summed up forefathers' work on microblogging platform in the last few years.By much-talked-about topic being found to the analysis of this proposition, this article selects this text-processing technology of semantic analysis as the basis of setting forth one's views, and has proposed an iterative semantic analysis and topic temperature forecast model---TopicRank in conjunction with the time series on microblogging platform and text feature.This model is divided by timeslice and two concepts of keyword set of topic are calculated in the influence power in continuous time section topic, thereby the influence power variation tendency within following a period of time is made a prediction to topic.This article has also been introduced the cyberrelationship that social networks structure in micro-blog the feature information based on user itself and the concern relation between user form and has been studied, and by the observation of this network proposed based on user role to topic temperature sequence carry out auxiliary method TopicRank-U.This article has provided a modular system prototype design, and by a series of test with experimental results show that the validity of two temperature order models that this system realizes.

" the network burst focus incident detection method based on topic model " (application publication number: CN102289487A, Shen Qing Publication order: 2011.12.21) disclose a kind of network burst focus incident detection method based on topic model.Comprise the steps: 1) first document data collection is carried out to word segmentation processing, obtain word list, document word relationship matrix, word Document distribution matrix, word date distribution matrix; 2) according to network focus incident, emerging in large numbers the burst characteristic of related words and document in process screens data set; 3) then by theme modeling obtain happening suddenly feature word and the feature text of focus incident; 4) calculate the attention rate date distribution of focus incident.

" network hot topic detection method and the device found based on Maximum Clique " (application publication number: CN102346766A, Shen Qing Publication day: 2012.02.08) disclose a kind of network hot topic detection method and device of finding based on Maximum Clique.Wherein, the method comprises the steps: Real-time Collection Internet news website, forum, blog, microblogging data; The data that gather are carried out to the processing such as participle, word frequency statistics, find all focus words pair, build focus words pair set and close; Each focus word is represented by unique numbering; Focus words pair set is closed and regarded non-directed graph as, it is excavated, obtain all Maximum Cliques; Each Maximum Clique is transformed to a word combination, represents a much-talked-about topic.A kind of network hot topic pick-up unit is also disclosed.

Generally speaking, in prior art, Hot spots detection complexity is high, a little less than practicality, is difficult in real time big data quantity be calculated.

Summary of the invention

The object of the invention is to, provide and can in social networks, to focus, carry out the means of real-time detection and classification.

According to an aspect of the present invention, provide a kind of real-time hot spot detecting method of delivering state for the network user.Method comprises step: the hot word of frequency detecting occurring according to word in the state of being delivered by user, wherein, when frequency that a word occurs being detected within the unit interval and occur saltus step, is identified as hot word by this word; Calculate and the maximally related a plurality of keywords of hot word; A plurality of keywords are aggregated into focus.

In an embodiment of the invention, when detecting hot word, calculate the impact energy of the function of the frequency occurring as word, and when impact energy is greater than threshold value, judges and occur saltus step.

In an embodiment of the invention, in the time window of schedule time length, calculate impact energy.

In an embodiment of the invention, when calculating a plurality of keyword, calculate the degree of correlation with the function of the state set that comprises other words as the state set that comprises hot word, and choose the most much higher other words of the degree of correlation as keyword.

In an embodiment of the invention, the focus that only retains nearest some.

In an embodiment of the invention, calculate the Euclidean distance of current focus and focus before, if distance is less than threshold value, current focus and focus are before merged.

In an embodiment of the invention, in the mode of time backward, calculate the Euclidean distance of current focus and focus before.

In an embodiment of the invention, according to participating user number, correlation behavior number, maximum propagation length, independent outburst source number, information entropy, social tight ness rating, position tight ness rating, user, explaining one or more in similarity classifies to focus.

In an embodiment of the invention, hotspot's classification is become to overall focus, region focus, popular focus.

Be different from prior art, according to various embodiments of the present invention, by applying according to the real-time hot spot detecting method of delivering state for the network user of the present invention, can reduce algorithm complex, strengthen practical, can be real-time big data quantity is calculated to (the status number > 50 that user per hour delivers, 000), and can be according to the characteristic in community network, utilize some exclusive features of available community network to carry out automatic classification to the focus detecting, and obtain very high accuracy rate, reasonable effect.

Accompanying drawing explanation

By shown embodiment is by reference to the accompanying drawings elaborated, above-mentioned and other features of the present invention will be more obvious, and in accompanying drawing of the present invention, identical reference number represents same or analogous element.In the accompanying drawings:

Fig. 1 be according to the embodiment of the present invention for the network user, deliver the process flow diagram of the real-time hot spot detecting method of state;

Fig. 2 is the time dependent schematic diagram of status number according to the embodiment of the present invention;

Fig. 3 is the time dependent schematic diagram of word frequency according to the embodiment of the present invention;

Fig. 4, Fig. 5 and Fig. 6 are the schematic diagram of hotspot's classification according to the embodiment of the present invention.

Embodiment

Below in conjunction with Fig. 1 set forth according to the embodiment of the present invention for the network user, deliver the real-time Hot spots detection scheme of state.

In step S102, the hot word of frequency detecting occurring according to word in the state of being delivered by user.When frequency that a word occurs being detected within the unit interval and occur saltus step, this word is identified as to hot word.Preferably, calculate the impact energy of the function of the frequency occurring as word, and when impact energy is greater than threshold value, judges and occur saltus step.Impact energy can be calculated in the time window of schedule time length.

In step S104, calculate and the maximally related a plurality of keywords of hot word.Preferably, calculate the degree of correlation with the function of the state set that comprises other words as the state set that comprises hot word, and choose the most much higher other words of the degree of correlation as keyword.

In step S106, a plurality of keywords are aggregated into focus.Preferably, calculate the Euclidean distance of current focus and focus before, if distance is less than threshold value, current focus and focus are before merged.Can calculate in the mode of time backward the Euclidean distance of current focus and focus before.

Because focus can change along with the time, so can only retain the focus of nearest some.

In addition can classify to focus.Classification foundation can include but not limited to that participating user number, correlation behavior number, maximum propagation length, independent outburst source number, information entropy, social tight ness rating, position tight ness rating, user explain similarity etc.Hotspot's classification can be become to overall focus, region focus, popular focus etc.

Describe real-time Hot spots detection and classification based on time window and the change of word frequency peak according to the embodiment of the present invention below in detail.

The target of embodiments of the present invention is to detect in real time explosive focus incident and the much-talked-about topic occurring in online social networks, and automatically the focus detecting is classified.

Input content is the state (for example: the state of Renren Network, the microblogging that in Sina's microblogging, user sends out) that SNS (Social Networking Services, social network services) user delivers in real time.

Focus incident and the classification (overall focus, region focus, popular topic etc.) thereof of output content for occurring in set time window.

First, carry out Hot spots detection.

By analyzing word frequency, change, the saltus step of identification word frequency (so-called word frequency is exactly the frequency that word occurs, the number of times occurring in the unit interval), detects hot word (spike).Take hot word as clue, calculate k the keyword relevant to its top (being exactly the most relevant, front k word of correlativity maximum), be polymerized to a focus.

It is a word that the common frequency of occurrences is higher that Fig. 2 illustrates " one ", can find that it presents cyclical variation with status number, more stable.And Fig. 3 illustrates " motor-car " since No. 23 21 left and right (take last Jul No. 23 Wenzhou motor-car accidents be example), present obvious impact (spike).

To the detection of hot word, can utilize model abstract.

State was divided by the time (15 minutes), be cut to different time windows, the state in each time window forms a collection of document.

User in each time window is delivered to all states and cut word, and set up inverted index, " impact " energy that defines each word is energy:

${\mathrm{energy}}_k^t=\frac{{\mathrm{<figure-callout\; id="1"\; label="ntf\; k\; t"\; filenames="HSA00000793890900021.png,HSA00000793890900031.png"\; state="\{\{state\}\}">ntf</figure-callout>}}_{kt}^{}}{1+{\mathrm{\&Sigma;}}_{x=1}^n{\mathrm{ntf}}_k^x/n}$

Wherein

(normalized term frequency, i.e. word frequency after standardization) represents the word frequency of word k (k is the subscript of energy, represents some words) in time window t after normalization here, that is: in N=time window t, delivered the number of users sum of state.If judge that k is as the hot word detecting, E is the threshold value of setting in advance.

The calculating of related term, definition

$r_{\mathrm{ij}}=\frac{\left|S_{i}n{S}_j\right|}{\left|S_{i}u{S}_j\right|}$

S wherein _irepresent the state set that comprises word i, calculate r _ki,i={1,2 ..n}, chooses topk and the maximally related word of k, is polymerized to a focus HS (hot spot).

Related term sorts by weight:

w _i＝ntf _i×idf _i

${\mathrm{idf}}_i=\log \left(\frac{\left|S\right|}{\left|S_i\right|}\right)$

W wherein _ibe the weight of i word, idf is inversed document frequency, i.e. inverse document frequency, | S| is all status numbers in time window t, | S _i| be the status number that comprises i word.

In time window t, any state that has comprised any two words in focus collection HS, is defined as and belongs to this focus.By the sequence of max-forwards number, choose and forward maximum states as representative (the represent)-i.e. description of this focus incident, topic of this focus.

Next, carry out hotspot tracking.

Retain nearest M={e ₁, e ₂... e _m(e represents event, event) individual focus incident, by time backward, calculate current focus e and e _ieuclidean distance, if Edist (e, e _i) < D (be Euclidean_distance, D is given threshold value, is taken as 1 in experiment), merge e and e _iit is a focus incident.Wherein Edist is the Euclidean distance between event:

$|d_j-d_k|=\sqrt{{\mathrm{\&Sigma;}}_{i=1}^n{(d_{i,j}-d_{i,k})}^2}$

Shape 20dj state and state d _kbetween distance, which word i represents

$\mathrm{sim}(\mathrm{<figure-callout\; id="1"\; label="d"\; filenames="HSA00000793890900021.png,HSA00000793890900031.png"\; state="\{\{state\}\}">d</figure-callout>}1,d2)=\frac{\underset{i}{\mathrm{\&Sigma;}}(a_i*b_i)}{\sqrt{\underset{i}{\mathrm{\&Sigma;}}{a}_i^2*\underset{i}{\mathrm{\&Sigma;}}{b}_i^2}}$

Because characterize the tf*idf vector of event and be after rule stroke, so Euclidean distance is the same with cosine similarity.In our system, get D=1, m=1000, threshold value is 1, preserves nearest 1000 focus incidents.

Advantageously, can carry out automatic classification.For example, referring to Fig. 4.

The focus detecting for each calculates its proper vector V=v number of users in testing process, maximum propagation length, and independent outburst source number, information entropy, social tight ness rating, location tight ness rating, compactness).

Number of users (userNum): the number that participates in this focus.

Status number (statusNum): the status number that this focus is relevant.

Maximum propagation length (max_cluster): the maximum user in current time diffusion collects size.

Independent outburst source number (busrtSrcNum): refer to independently deliver the number of users that this focus is relevant (not being to forward).

Information entropy:

p _iit is the probability that i word occurs.

Social tight ness rating:

e _ifor participating in all users of this focus, the good friend limit number of existence, the total number of users of n for participating in.

Location tight ness rating:

n _irepresent school of a certain institute or some cities participate in this focus subscriber data in the number of times that occurs, n is total number of users.

User explains similarity: $\frac{{\mathrm{\&Sigma;}}_{i=1}^n{\mathrm{\&Sigma;}}_{j=i+1}^n\mathrm{Dist}(d_i.d_{j)}}{n\&times;(n-1)/2},$ D _iand d _jrepresent User Status i and state j, Dist represents Euclidean distance, and n represents to participate in total status number of this focus.

Mark training sample, with svm sorter to focus classify (overall situation is broken out focus, region focus, popular topic).

Be with the difference of prior art maximum, technical scheme complexity of the present invention is low, real-time is good, practical, can be real-time big data quantity is calculated to (the status number > 50 that user per hour delivers, 000), and according to the characteristic in community network, utilize some exclusive features of available community network (such as independent outburst source, maximum propagation length, social tight ness rating) to carry out automatic classification to the focus detecting, and obtain very high accuracy rate, reasonable effect.

It should be noted that embodiments of the present invention can realize by the combination of hardware, software or software and hardware.Hardware components can utilize special logic to realize; Software section can be stored in storer, and by suitable instruction execution system, for example microprocessor or special designs hardware are carried out.The equipment the present invention relates to and module thereof can be by such as VLSI (very large scale integrated circuit) or gate array, realize such as the semiconductor of logic chip, transistor etc. or such as the hardware circuit of the programmable hardware device of field programmable gate array, programmable logic device etc., also can use the software of being carried out by various types of processors to realize, also can by the combination of above-mentioned hardware circuit and software for example firmware realize.

In addition, although described in the accompanying drawings the operation of the inventive method with particular order,, this not requires or hint must be carried out these operations according to this particular order, or the operation shown in must carrying out all could realize the result of expectation.On the contrary, the step of describing in process flow diagram can change execution sequence.Additionally or alternatively, can omit some step, a plurality of steps be merged into a step and carry out, and/or a step is decomposed into a plurality of steps carries out.

Although described the present invention with reference to some embodiments, should be appreciated that, the present invention is not limited to disclosed embodiment.The present invention is intended to contain interior included various modifications and the equivalent arrangements of spirit and scope of appended claims.The scope of appended claims meets the most wide in range explanation, thereby comprises all such modifications and equivalent structure and function.

Claims (9)

1. for the network user, deliver a real-time hot spot detecting method for state, comprise step:

The hot word of frequency detecting occurring according to word in the state of being delivered by user, wherein, when frequency that a word occurs being detected within the unit interval and occur saltus step, is identified as hot word by this word;

Calculate and the maximally related a plurality of keywords of described hot word;

Described a plurality of keywords are aggregated into focus.

2. real-time hot spot detecting method according to claim 1, wherein, when detecting hot word, calculates the impact energy of the function of the frequency occurring as word, and when described impact energy is greater than threshold value, judges and occur saltus step.

3. real-time hot spot detecting method according to claim 2 wherein, calculates impact energy in the time window of schedule time length.

4. real-time hot spot detecting method according to claim 1, wherein, when calculating a plurality of keyword, calculate the degree of correlation with the function of the state set that comprises other words as the state set that comprises described hot word, and choose the most much higher described other words of the degree of correlation as keyword.

5. real-time hot spot detecting method according to claim 1, wherein, only retains the focus of nearest some.

6. according to the real-time hot spot detecting method described in any one in claim 1-5, wherein, calculate the Euclidean distance of current focus and focus before, if described distance is less than threshold value, by described current focus with described before focus merging.

7. real-time hot spot detecting method according to claim 6, wherein, calculates the Euclidean distance of current focus and focus before in the mode of time backward.

8. according to the real-time hot spot detecting method described in any one in claim 1-5, wherein, according to participating user number, correlation behavior number, maximum propagation length, independent outburst source number, information entropy, social tight ness rating, position tight ness rating, user, explaining one or more in similarity classifies to focus.

9. real-time hot spot detecting method according to claim 8, wherein, becomes overall focus, region focus, popular focus by hotspot's classification.

Images