CN105243149A - Semantic-based query recommendation method and system - Google Patents

Abstract

The invention provides a semantic-based query recommendation method and system. The method comprises the steps of establishing a query concept binary diagram, establishing a query click URL binary diagram, combining the query concept binary diagram with the query click URL binary diagram according to query nodes to form a concept query click URL ternary diagram, and establishing three layers of random walk models; according to query terms input by users, calculating the walk possibility between the input query term nodes, and the concept nodes and the URL nodes based on the three layers of random walk models, arranging the input query term nodes from high to low based on the walk possibility to obtain a query recommendation list. The semantic-based query recommendation method and system provided by the invention aim to enhance the semantic relevance between the recommended query and the original query to recommend better queries for users so as to meet the query demands of users.

Description

A kind of semantic-based web query recommend method and system

Technical field

The present invention relates to information retrieval field, particularly a kind of semantic-based web query recommend method and system.

Background technology

Along with the development in epoch, internet also worldwide obtains vigorous growth, and increasing people starts to look for required information on the internet.Along with the information on internet gets more and more, search engine becomes the requisite instrument of customer access network resource gradually, and user, by search engine input inquiry, reflects the information requirement of oneself, search engine obtains and analyzes user's inquiry, returns to the information that user needs.But be not inquiry each time, user can obtain the result wanted.This is because user is when inquiring about some information on the one hand, inherently this lack of knowledge understanding on the one hand, is difficult to construct suitable inquiry; On the other hand, due to search engine technique reasons, search engine self sometimes also can the inquiry of confusing user; Finally, due to the polysemy of natural language, some inquiry originally has different semantemes under different context.Such as, as user's input " apple ", user wants to search for apple scientific & technical corporation, and user wants to search for apples, and user thinks that the film of " apple " is in search one.Due to above various reasons, search engine is difficult to ensure just can to selling customer satisfaction system result when user's first time input.In order to understand the query intention of user better, help the inquiry that user constructs, modern search engines both provides the technology that inquiry is recommended.

The method that the inquiry of current main flow is recommended has three kinds: a kind of is method based on document, and these class methods are by process query search document out, in this, as feedback, understand user view further, expand the semanteme of inquiry.The second is the method based on inquiry log, is to utilize inquiry log to obtain similar inquiry, utilizes the user profile of inquiry log record, can excavate query word and directly contact.The third is conversation-based method, first to conversate division to search daily record, then utilizes correlation rule to each session, correlativity between mutual information and similarity algorithm metrics query.

At present, Peking University proposes a kind of carrying out based on user journal and inquires about the method and system of recommending, and obtains the inquiry log of being correlated with according to the data set in user journal, and selects some query strings to carry out inquiry recommendation as training set; The inquiry that International Business Machine Corporation (IBM) inputs according to user, obtains corresponding inquiry and recommends set to recommend from different search engines; Company of Baidu utilizes user grouping to recommend; Company of Baidu also builds inquiry by user gradation and recommends, the method is according to the interactive problem of the query generation of user, and interactive problem is provided to user, according to user, the answer of interactive problem is determined to the grade of user, finally recycle the grade of user and search word generated query recommendation results and be provided to user.

Although inquiry recommended technology is comparatively ripe, still there are some problems:

The first, the method based on document needs outside document, and the time complexity that global document is analyzed is high, limit by natural language processing technique, effect neither be fine, and local document analysis is for how Obtaining Accurate inquiry relevant documentation is also a difficult problem, and computing cost is still very large.

The second, conversation-based method needs first to carry out sessionizing accurately to inquiry log, and session is more less than inquiry, and direct basis statistical information still also exists the sparse problem of information.Method based on click information then relies on the effect of search engine very much, and user clicks in result and also there is many noises and skewed popularity, and the inquiry in inquiry log also also exists openness problem.

3rd, traditional inquiry recommend method is generally only considered click information or is query word itself, seldom can take into account the Deep Semantics after query word.This causes the result of recommending out to there is the not high situation of accuracy.

4th, openness due to inquiry log, the number of times that a lot of query word occurs in inquiry log does not seldom even all occur.For these inquiries, the inquiry that traditional method is also difficult to provide is recommended.

Summary of the invention

For above problem, patent object of the present invention is to devise a kind of semantic-based web query recommend method and system, is intended to the semantic relevance strengthening inquiry and the original query of recommending out, recommends out better inquiry to user, meet the query demand of user.

The present invention is achieved by the following technical solutions:

The invention provides a kind of semantic-based web query recommend method, comprising:

Obtain historical query word according to user's historical query daily record data, historical query word is mapped to wikipedia concept, set up query concept binary pattern;

According to user's historical query daily record data, historical query daily record is mapped with click URL, builds inquiry and click URL binary pattern, by the historical query of user with click behavior record and click in URL binary pattern in inquiry;

Described query concept binary pattern and inquiry are clicked URL binary pattern merge according to query node, form concept queries and click URL ternary diagram, and set up three layers of random walk model;

According to the input inquiry word of user, utilize described three layers of random walk model to calculate input inquiry word node and the migration probability between concept node and URL node, input inquiry word node is arranged from high to low according to migration probability, obtain inquiring about recommendation list.

Further, of the present invention historical query word is mapped to wikipedia concept, comprises further:

By the inverted index that wikipedia document is weighted, constructing semantic interpreter;

Semantic interpreter is utilized historical query word to be mapped to concept in wikipedia.

Further, historical query word is mapped to concept in wikipedia by the semantic interpreter that utilizes of the present invention, comprises further:

Historical query word is carried out participle, and each participle carries out corresponding with wikipedia document according to TF-IDF value.

Further, TF-IDF value computing formula of the present invention is as follows:

$TF-IDF={\mathrm{TF}}_{ij}\×{\mathrm{IDF}}_i=\frac{n_{ij}}{{\Σ}_k{n}_{kj}}\×log\frac{\left|D\right|}{\left|\right\{j:t_i\∈d_j\left\}\right|+1}$

Wherein, TF-IDF value represents the number of times that word i occurs in document j, n _ijthe number of times that word i occurs in document j, n _kjit is the occurrence number of all words in document j; | D| is the total number of files in corpus, | { j:t _i∈ d _j| for containing the document number of word i.

Further, of the present invention click on URL ternary diagram in described concept queries calculates query node and the migration probability between concept node and URL node, and computing formula is as follows:

$P_{ij}=C_{ij}/\underset{k\∈B}{\Σ}{C}_{ik}$

P _ijbe the probability that node i one step in a category node A transfers to a node j in another kind of Node B, C _ijfor the weights that node i is connected with node j.

Further, user's historical query daily record data of the present invention comprises the URL of user's name information, user's query contents information, the URL of click, the time of inquiry and click.

Further, three layers of random walk model of the present invention are three layers of Markov random walk model.

The present invention also provides a kind of semantic-based web query commending system, comprising:

Conceptual Projection module, is used for obtaining historical query word according to user's historical query daily record data, historical query word is mapped to wikipedia concept, sets up query concept binary pattern;

Inquiry and click URL respective modules, be used for according to user's historical query daily record data, historical query daily record be mapped with click URL, build inquiry and click URL binary pattern;

Three layers of random walk module, are used for that described query concept binary pattern and inquiry are clicked URL binary pattern and merge according to query node, form concept queries and click URL ternary diagram, and set up three layers of random walk model;

User's load module, is used for inputting user's query word;

Inquiry recommending module, be used for utilizing described three layers of random walk model to calculate input inquiry word node and the migration probability between concept node and URL node, input inquiry word node arranged from high to low according to migration probability and obtains inquiry recommendation list inquiry recommendation is carried out to user.

A kind of semantic-based web query recommend method provided by the invention and system, by query mappings being become the concept of wikipedia, extend the Semantic of inquiry, and it is combined with click URL information, utilize three layers of random walk model, inquiry recommendation results is sorted from high to low according to migration probability, recommendation results is made all to keep correlativity with original inquiry in semantic information and click URL information, finally more accurate, correlativity is recommended out better to inquire about, for user, improve the satisfaction of user.

Accompanying drawing explanation

Referring to accompanying drawing, embodiments of the present invention is further illustrated, wherein:

Fig. 1 is the query concept binary pattern of a kind of semantic-based web query recommend method of the present invention;

Fig. 2 is that URL binary pattern is clicked in the inquiry of a kind of semantic-based web query recommend method of the present invention;

Fig. 3 is that the concept queries of a kind of semantic-based web query recommend method of the present invention clicks URL ternary diagram;

Fig. 4 is the module map of a kind of semantic-based web query commending system of the present invention.

Embodiment

Below in conjunction with the drawings and specific embodiments, the present invention is described in further detail.

The present invention proposes and the invention provides a kind of semantic-based web query recommend method, comprise the steps:

Step (A), obtains historical query word according to user's historical query daily record data, historical query word is mapped to wikipedia concept, set up query concept binary pattern.

Concrete, query word is often very short, generally has 2,3 word compositions, expands the semanteme of query word, overcome the shortcoming of the semanteme sparse due to query word by concept.Conceptual Projection process has mainly used the display semantic analysis technology based on wikipedia, query word has been mapped to the concept in wikipedia.The concept of wikipedia here refers to the entry of wikipedia, and the concrete steps of mapping are as follows:

(1) wikipedia document is utilized to build semantic interpreter;

(2) semantic interpreter is utilized query word to be mapped to concept in wikipedia.

Step (1) utilizes wikipedia document to build semantic interpreter, for the inverted index having weight set up by these wikipedia documents.So-called inverted index is the concrete file layout of one realizing " word-document matrix ", by inverted index, and can according to the lists of documents of this word of word quick obtaining.It is primarily of two part compositions: word lexicon and inverted file.Word lexicon is worth the string assemble be made up of all words occurred in collection of document, the string assemble formed in word lexicon.Inverted file is the place of the Inverted List recording all words, and then describe the positional information that the lists of documents of all documents occurring certain word and word occur in a document in Inverted List, every bar record is called one and arranges item.According to inverted list, which document package can be obtained and contain certain word.

Step (2) is the concept with semantic interpreter, query word being mapped to wikipedia.First query word is carried out participle, participle refers to and query word is divided into word independent one by one, and each word can find wikipedia document corresponding with it in inverted index in step (1), and the weights of their correspondences can be determined according to TF-IDF value.

TF-IDF (termfrequency-inversedocumentfrequency) is a kind of statistical method, in order to assess the significance level of a words for document sets or one section of document.The number of times that importance and it of words occur in a document is relevant, appearance more much more important, and this is also the meaning of TF (termfrequency, word frequency), and the TF value computing formula of word i in document j is as follows:

${\mathrm{TF}}_{ij}=\frac{n_{ij}}{{\Σ}_k{n}_{kj}}$

N _ijthe number of times that word i occurs in document j, n _kjit is the occurrence number of all words in document j.

But some word is the word such as " ", " good " such as, almost occurred all frequently many times in each section of document, the significance level of these words should reduce, Here it is IDF (inversedocumentfrequency, reverse document frequency) meaning, what its calculated is the inverse ratio of the frequency that a word occurs in corpus.For word i, its IDF is calculated as follows:

${\mathrm{IDF}}_i=log\frac{\left|D\right|}{\left|\right\{j:t_i\∈d_j\left\}\right|+1}$

| D| is the total number of files in corpus, | { j:t _i∈ d _j| for containing the document number of word i.

In order to not allow denominator be 0, so add one 1 later.And TF-IDF value is TF value and IDF is worth product, i.e. TF-IDF=TF*IDF.So a word i in the TF-IDF value of document j is:

TF-IDF _ij＝TF _ij*IDF _i

And an inquiry is made up of several words, the TF-IDF Data-Statistics that they are all gets up, and just obtains the corresponding relation of inquiry and wikipedia document.And wikipedia document is and wikipedia concept (i.e. the entry of wikipedia) one_to_one corresponding, inquiry has been mapped with the concept of wikipedia by we like this, and can calculate weight corresponding between them.

Can advise setting up a query concept binary pattern according to these, refer to Fig. 1, have concept cluster node V1 and inquiry category node V2 in query concept binary pattern, the line between two category nodes is exactly the TF-IDF weights between them.

Step (B), according to user's historical query daily record data, is mapped historical query daily record with click URL, builds inquiry and click URL binary pattern, by the historical query of user with click behavior record and click in URL binary pattern in inquiry.

Concrete, inquiry log is the daily record that search engine generates in order to the search behavior of recording user, and it describes rank etc. the important information of user ID, user's inquiry, the URL clicked, the time of inquiry, the URL (URL(uniform resource locator)) of click.For the record in each inquiry log, one can be found to inquire about and the URL corresponding with it.Utilize historical query daily record, inquiry and click URL are mapped, build inquiry and click URL binary pattern, refer to Fig. 2, the node V in inquiry click URL binary pattern forms V1 and V2 by two large category nodes.V1 represents all inquiries, and V2 represents all URL.If the U1 in some Q1 and the V2 in V1 directly exists line, after representing that user inputs Q1, click U1.The weight e (Q1, U1) of line is relevant to URL the number clicked.

The building process that URL binary pattern is clicked in inquiry is as follows:

Inquiry is clicked URL binary pattern and is started to be set to sky;

For the record of in historical query daily record, take out its inquiry (Qi) and click URL (Uj);

For the inquiry (Qi) of to take out for every a pair with click URL (Uj), if click not inquiry (Qi) in URL binary pattern in inquiry, add query node (Qi); If inquiry is clicked in URL binary pattern do not have URL (Uj), add URL node (Uj); If Qi and Uj does not have line, then coupled together by Qi and Uj, the weight of juxtaposition limit eij is 1; If have line between Qi and Uj, then the weight of limit eij is added 1;

Repeat second step to the 3rd step until all records of inquiry log have all been processed.

Step (C), clicks URL binary pattern by described query concept binary pattern and inquiry and merges according to query node, forms concept queries and clicks URL ternary diagram, and set up three layers of random walk model.

Concrete, the query node that the inquiry that step (A) and step (B) obtain is clicked in URL binary pattern and query concept binary pattern is one to one, directly two figure are carried out merging according to query node, concept queries can be obtained and click URL ternary diagram, refer to Fig. 3.

In concept queries URL ternary diagram, and calculate migration probability on this ternary diagram, ternary diagram is become three layers of Markov random walk model.Be made up of concept, inquiry, URl tri-category node, wherein inquire about category node can a step migration to concept cluster node and URL category node, and between concept cluster node and URL category node cannot one step transfer arrival.

Migration probability calculation process between inquiry category node and other two category nodes is as follows:

$P_{ij}=C_{ij}/\underset{k\∈B}{\Σ}{C}_{ik}$

P _ijbe the probability that node i one step in a category node A transfers to a node j in another kind of Node B, C _ijfor the weights that node i is connected with node j.

Utilize this computing formula, arbitrfary point in concept queries URL ternary diagram can be calculated and set out, transfer to the probability of other nodes through t step, thus we can obtain transition probability matrix.Row and column in transition matrix (being denoted as Z) is all the node in ternary diagram, and the value of each unit is the probability shifted between them.The transition probability transferring to node j from ternary diagram interior joint i is:

P _ij＝[Z ^t] _ij

Click in URL tri-layers of random walk model in concept queries, i.e. three layers of Markov random walk model, from certain query node, after the random walk of even number step, finally according to probability migration in inquiry category node.These query nodes are arranged from high to low according to migration probability, just can obtain inquiring about recommendation list.

Step (D), according to the input inquiry word of user, user input query, usual use browser, as conventional InternetExplorer, FireFox, the browsers etc. such as Chrome, utilize described three layers of random walk model to calculate input inquiry word node and the migration probability between concept node and URL node, input inquiry word node is arranged from high to low according to migration probability, obtain inquiring about recommendation list.

Accordingly, the present invention also provides a kind of semantic-based web query commending system, refers to Fig. 4, comprising:

Conceptual Projection module, is used for obtaining historical query word according to user's historical query daily record data, historical query word is mapped to wikipedia concept, sets up query concept binary pattern;

Inquiry and click URL respective modules, be used for according to user's historical query daily record data, historical query daily record be mapped with click URL, build inquiry and click URL binary pattern;

Three layers of random walk module, are used for that described query concept binary pattern and inquiry are clicked URL binary pattern and merge according to query node, form concept queries and click URL ternary diagram, and set up three layers of random walk model;

User's load module, is used for inputting user's query word;

Inquiry recommending module, be used for utilizing described three layers of random walk model to calculate input inquiry word node and the migration probability between concept node and URL node, input inquiry word node arranged from high to low according to migration probability and obtains inquiry recommendation list inquiry recommendation is carried out to user.

A kind of semantic-based web query recommend method provided by the invention and system specific embodiment, inquiry commending system is divided into line upper part and line lower part two, and line lower part can process by off-line, and line upper part then needs to process online.

In line lower part, Conceptual Projection unit, first by the inverted index that wikipedia document is weighted, is configured to query semantics interpreter.So-called semantic interpreter, can be mapped to the concept of wikipedia exactly by content of text.Then utilize inquiry log to obtain the query word of all records, for each query word, it is carried out participle by Conceptual Projection unit, obtains several independently words, and the wikipedia document sets then putting into index good carries out index, obtains TF-IDF value.

Dimension base document sets is carried out inverted index, then for each inquiry in inquiry log, divide good word, be put into the calculating carrying out TF-IDF value in the good dimension base document sets of index, and statistics is added, to be inquired about and each tie up the probability contacted between base document, and the document of wikipedia is one to one with tieing up base concept (that is to say the entry of wikipedia), we are just mapped inquiry and the unification of dimension base concept like this, and then construct query concept binary pattern.

Inquiry is clicked URL map unit and is utilized inquiry log, will inquire about and click URL mutually corresponding.Every bar record in inquiry log, one can be found to inquire about and the click URL corresponding with it, they reflect search, the click behavior of user.In general, if the click URL of two inquiry correspondences distributes very similar, so illustrate that these two inquiries are also comparatively similar.Inquiry is clicked URL map unit and is passed through process inquiry log, builds inquiry, clicks URL binary pattern, the critical data of the search of user, click behavior has been recorded in this binary pattern.

After obtaining inquiry click URL binary pattern and query concept binary pattern, these two binary patterns merge by three layers of random walk unit, form concept queries and click URL ternary diagram, and calculate migration probability on this ternary diagram, ternary diagram is become three layers of Markov random walk model.Construct concept queries and click URL tri-layers of random walk model.

Line upper part mainly processes the inquiry of user's input online, Random Walk Algorithm is applied by three layers of random walk model, provides and inquire about recommendation accordingly.User input unit is responsible for the inquiry accepting user's input, when user input unit has got inquiry, can retrieve on the one hand, provide result for retrieval.On the other hand, inquiry is put in three layers of random walk model of off-line generation by it, carries out inquiring about the work recommended.

Click in URL ternary diagram in the concept queries of three layers of random walk model, using this inquiry as the start node of random walk, in these three layers of random walk models, carry out t walk random walk, the value of transition matrix Z after the migration of calculating transfer t step.Suppose that the node of original query in three layer model is i, after so t walks migration, calculate Z ^t, final Z ^tthe value of the i-th row be exactly transition probability value to each query node.By final transition probability according to sorting from big to small, selecting a front n result and recommending as inquiry.

The above the specific embodiment of the present invention, does not form limiting the scope of the present invention.Various other that any technical conceive according to the present invention is made change and distortion accordingly, all should be included in the protection domain of the claims in the present invention.

Claims (8)

1. a semantic-based web query recommend method, is characterized in that, comprising:

Obtain historical query word according to user's historical query daily record data, historical query word is mapped to wikipedia concept, set up query concept binary pattern;

According to user's historical query daily record data, historical query daily record is mapped with click URL, builds inquiry and click URL binary pattern, by the historical query of user with click behavior record and click in URL binary pattern in inquiry;

Described query concept binary pattern and inquiry are clicked URL binary pattern merge according to query node, form concept queries and click URL ternary diagram, and set up three layers of random walk model;

According to the input inquiry word of user, utilize described three layers of random walk model to calculate input inquiry word node and the migration probability between concept node and URL node, input inquiry word node is arranged from high to low according to migration probability, obtain inquiring about recommendation list.

2. semantic-based web query recommend method according to claim 1, is characterized in that, described historical query word is mapped to wikipedia concept, comprises further:

By the inverted index that wikipedia document is weighted, constructing semantic interpreter;

Semantic interpreter is utilized historical query word to be mapped to concept in wikipedia.

3. semantic-based web query recommend method according to claim 2, is characterized in that, historical query word is mapped to concept in wikipedia by the described semantic interpreter that utilizes, and comprises further:

Historical query word is carried out participle, and each participle carries out corresponding with wikipedia document according to TF-IDF value.

4. semantic-based web query recommend method according to claim 3, is characterized in that, described TF-IDF value computing formula is as follows:

$TF-IDF={\mathrm{TF}}_{ij}\×{\mathrm{IDF}}_i=\frac{n_{ij}}{{\mathrm{\Σ}}_k{n}_{kj}}\×log\frac{\left|D\right|}{\left|\right\{j:t_i\∈d_j\left\}\right|+1}$

Wherein, TF-IDF value represents the number of times that word i occurs in document j, n _ijthe number of times that word i occurs in document j, n _kjit is the occurrence number of all words in document j; | D| is the total number of files in corpus, | { j:t _i∈ d _j| for containing the document number of word i.

5. semantic-based web query recommend method according to claim 1, is characterized in that, described click on URL ternary diagram in described concept queries calculates query node and the migration probability between concept node and URL node, and computing formula is as follows:

$P_{ij}=C_{ij}/\underset{k\∈B}{\Σ}{C}_{ik}$

P _ijbe the probability that node i one step in a category node A transfers to a node j in another kind of Node B, C _ijfor the weights that node i is connected with node j.

6. semantic-based web query recommend method according to claim 1, is characterized in that, described user's historical query daily record data comprises the URL of user's name information, user's query contents information, the URL of click, the time of inquiry and click.

7. semantic-based web query recommend method according to claim 1, is characterized in that, described three layers of random walk model are three layers of Markov random walk model.

8. a semantic-based web query commending system, is characterized in that, comprising:

Conceptual Projection module, is used for obtaining historical query word according to user's historical query daily record data, historical query word is mapped to wikipedia concept, sets up query concept binary pattern;

Inquiry and click URL respective modules, be used for according to user's historical query daily record data, historical query daily record be mapped with click URL, build inquiry and click URL binary pattern;

Three layers of random walk module, are used for that described query concept binary pattern and inquiry are clicked URL binary pattern and merge according to query node, form concept queries and click URL ternary diagram, and set up three layers of random walk model;

User's load module, is used for inputting user's query word;

Inquiry recommending module, be used for utilizing described three layers of random walk model to calculate input inquiry word node and the migration probability between concept node and URL node, input inquiry word node arranged from high to low according to migration probability and obtains inquiry recommendation list inquiry recommendation is carried out to user.