CN106294662A - Inquiry based on context-aware theme represents and mixed index method for establishing model - Google Patents

Abstract

The invention discloses a context-aware topic-based query representation and hybrid retrieval model building method, including the following steps: Step 1: Based on the keyword set of the query, obtain the pseudo-relevant feedback documents of the query, and select the pseudo-relevant feedback documents from the pseudo-related feedback documents Query the relevant context; Step 2: Introduce the context-aware topic model, integrate the context into the context-aware topic model, mine the topic information hidden in the context window based on the corpus topic, and obtain its corresponding topic vector; Step 3: Convert the query to the topic Vectors and keyword sets are jointly represented, and based on topic vectors and keyword sets, a hybrid retrieval model is established to obtain the final retrieval score.

Description

Query Representation and Hybrid Retrieval Model Building Method Based on Context-Aware Topics

technical field

The invention relates to the technical field of Internet information retrieval, in particular to a method for establishing a query representation and a hybrid retrieval model based on a context-aware topic model.

Background technique

Query representation has always been the core of the field of information retrieval, and the most common problem is that the user query is too short (contains only a few keywords), and it is easy to cause the relevant documents in the retrieval process to not match the query. For example, for the user query "short of water", if the document contains words related to the query such as "drought", although the correlation is high, but because the original query keyword "short of water" is not included, the final matching degree will be very low. low, thereby affecting the accuracy of the query.

A common solution is query expansion based on pseudo-relevance feedback. This method is based on the preliminary retrieval results, assuming that the top K documents (referred to as "pseudo-relevant feedback documents") are related to the original query, and the keywords in them can be extracted by using a correlation algorithm for query expansion. express. However, this method is unsupervised, and it is easy to bring some words irrelevant to the query. Although in theory, a supervised classification method can be used to comprehensively consider the various characteristics of the expanded words to select the words that are really related to the query. However, this method relies on feature engineering and labeled training sets, and the cost of practical application is high.

Some recent studies have focused on how to exploit various contextual information to alleviate the problem of irrelevant expansion word introduction in query representation. Sources of contextual information mainly include high-quality external data sources (such as encyclopedias, domain ontologies, etc.) and pseudo-relevant feedback documents based on the dataset itself. The former is not widely used in practice because it is only applicable to some queries, and external data sources are often updated slowly and difficult to obtain. The latter, based on the pseudo-relevant feedback documents of the dataset itself, actually also provides a contextual description of the query, which has a greater research prospect. For example, for the query "water shortage", pseudo-relevant feedback document 1 describes: "The UK will face water shortages in the next few years, so please save water and fix your taps."; pseudo-relevant feedback document 2 describes: " Rainfed agriculture: a way to mitigate drought and water scarcity". These two articles are all about the response to the water shortage problem, and these contextual information can be used to assist query representation. However, the existing extended word selection methods generally only consider the co-occurrence of the extended word and the original query word in the context window of the pseudo-relevance feedback, and there are still the following problems: (1) It is necessary to explicitly select which words to use as the final query Expansion, some irrelevant words, even "harmful words" will still be introduced in the unsupervised situation. For example, in articles involving various environmental resources, the keyword "water shortage" appears frequently, but similar words such as "hydroelectric power" and "natural gas" also appear in its context, which will deviate from the original query and reduce the accuracy of the query. (2) The final query representation is still based on the dictionary space, ignoring the semantic information implicit in the query, such as potential topics; (3) The retrieval model based on this query representation mainly considers keyword matching, but ignores the relationship between documents and queries. Matching at the semantic level.

Contents of the invention

The purpose of the present invention is to propose a query representation and hybrid retrieval model design method based on the context-aware topic model in view of the deficiencies of the prior art. Based on the word matching retrieval model, topic matching is added to improve the accuracy of retrieval results.

The present invention proposes a query representation based on context-aware topics and a method for establishing a hybrid retrieval model, which includes the following steps:

Step 1: Obtain a pseudo-relevant feedback document of the query based on the keyword set of the query, and select a context related to the query from the pseudo-relevant feedback document;

Step 2: introducing a context-aware topic model, integrating the context into the context-aware topic model, mining the topic information implied by the context window based on the corpus topic, and obtaining its corresponding topic vector;

Step 3: The query is jointly represented by the topic vector and the keyword set; based on the topic vector and the keyword set, a hybrid retrieval model is established to obtain a final retrieval score.

In the context-aware topic-based query representation and hybrid retrieval model building method proposed by the present invention, in step 1, the pseudo-relevance feedback document is divided into a plurality of sliding windows, and the correlation between each window and the query is calculated. Relevance, take the window whose correlation is higher than the threshold as the context window related to the query.

In the context-aware topic-based query representation and hybrid retrieval model building method proposed by the present invention, the query-related context selection threshold is the average value of all window correlations under the query.

In the context-aware topic-based query representation and hybrid retrieval model building method proposed by the present invention, the context-aware topic model is designed according to the relevant context of the query and the entire corpus, and the context-aware topic model is used in the topic modeling process Assuming that the context window and the pseudo-relevant feedback document where it is located share the same topic distribution, the topic vector of the context is obtained.

In the context-aware topic-based query representation and hybrid retrieval model building method proposed by the present invention, the pseudo-relevant feedback document is obtained by calculating the keyword matching score of the retrieval model.

In the context-aware topic-based query representation and hybrid retrieval model building method proposed by the present invention, the retrieval score is represented by the following formula:

$\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; \&lambda;</span>}<span\; class="notranslate">\; )</span>{<span\; class="notranslate">\; \&Sigma;</span>}_{{\mathrm{<span\; class="notranslate">\; q</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; \&Element;</span>\mathrm{<span\; class="notranslate">\; Q</span>}}\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; q</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; d</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&lambda;</span>}<span\; class="notranslate">\; \&Center\; Dot;</span>{\mathrm{<span\; class="notranslate">\; the\; s</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; Q</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; d</span>}<span\; class="notranslate">\; )</span>$

Among them, s represents the score based on keyword matching in the traditional retrieval model, s' represents the topic matching score based on the new query representation Q', and λ is the weight parameter between these two scores, and it is also the trade-off coefficient of the two matching methods.

The beneficial effect of the present invention is that: the present invention makes full use of the context information of the corpus itself based on pseudo-correlation feedback, and solves the problem that high-quality external data sources are difficult to obtain. And by dividing the pseudo-relevant feedback documents into context windows, and selecting context fragments that are more relevant to the query for query representation, the introduction of "noise" and query drift is reduced, which is an innovation of query representation quality control move. The context-aware topic model proposed in the present invention fully excavates the topic information corresponding to the context related to the query, breaks through the traditional understanding based only on the keyword level, and helps to understand user queries more comprehensively and deeply. Traditional retrieval models are mainly based on keyword matching, while ignoring the deep semantic correlation. The hybrid retrieval model designed by the present invention takes keyword matching and topic matching into consideration, and this diversified matching mode helps to promote the improvement of retrieval effect. The query representation method and hybrid retrieval model proposed by the present invention have been proved to be effective on the Microblog Track 2011-2014 data set, and the context topic information is integrated into the query, and the MAP value of the final retrieval exceeds some of the latest query representation methods .

Description of drawings

FIG. 1 is a flow chart of the context-aware topic-based query representation and hybrid retrieval model building method of the present invention.

Fig. 2 is a flowchart of context selection based on pseudo-correlation feedback.

Figure 3 is a graphical representation of a context-aware topic model.

detailed description

The present invention will be further described in detail in conjunction with the following specific embodiments and accompanying drawings. The process, conditions, experimental methods, etc. for implementing the present invention, except for the content specifically mentioned below, are common knowledge and common knowledge in this field, and the present invention has no special limitation content.

As shown in Figure 1, the query representation and hybrid retrieval model building method based on context-aware topics of the present invention includes the following steps:

Step 1: Based on the keyword set of the query, obtain the pseudo-relevant feedback document of the query, and select the context related to the query from the pseudo-related feedback document;

Step 2: Introduce the context-aware topic model, integrate the context into the context-aware topic model, mine the topic information hidden in the context window based on the corpus topic, and obtain its corresponding topic vector;

Step 3: The query is represented by the subject vector and the keyword set; based on the subject vector and the keyword set, a hybrid retrieval model is established to obtain the final retrieval score.

(1) Relevant context selection based on pseudo-relevant feedback

Since the pseudo-relevant feedback documents are easy to obtain and contain a lot of content related to the query, the present invention will select contexts that are relatively relevant to the query for query representation. The specific process is shown in Figure 2.

First, the pseudo-relevant feedback documents are segmented to obtain multiple context windows of size n. Define Q={q ₁ , q ₂ , . . . , q _|Q| } as a query, where q _i represents a query keyword, and |Q| represents the number of keywords in the query. is the set of pseudo-relevant feedback documents corresponding to the query Q, that is, the documents ranked top k in the first retrieval. For a pseudo-relevant feedback document In the form of a sliding window, it will be divided into several context windows of size n (including n words) as shown in Figure 2, that is, Q _c1 , Q _c2 ,..., Q _cl , and I represents the context window Number of.

Second, the relevance of the context window to the original query is calculated. For a query and context window pair (Q, Q _c ), the present invention uses a variety of methods to calculate the correlation R(Q, Q _c ) between them, such as the average point mutual information (Pointwise Mutual Information) based on word co-occurrence Information), the Jaccard similarity based on the word set, the semantic similarity based on the word vector word2vec, etc., and finally take the average value.

Then, filter out contexts that are relevant to the query. First, normalize the correlation obtained above. Next, set the threshold as the average of the correlations of all windows under the query, and filter out the context windows whose correlations are lower than the threshold, and the remaining contexts that are more relevant to the query will be further used for context-aware topic modeling.

(2) Contextual topic-aware modeling and query representation

Given the query-related context and the entire corpus obtained in (1), the present invention designs a context-aware topic model to incorporate query-related context information into the topic model to generate new query representations.

Inspired by related studies, since both the context window selected in (1) and the pseudo-relevant feedback document where it is located are closely related to the query, it is assumed that they share the same topic distribution. Under this assumption, the traditional LDA topic model is improved to obtain the context-aware topic model CAT, and its graphical model is shown in Figure 3. The relevant symbols involved in the model are shown in Table 1. The model is a generative model, and the specific modeling process is shown in Algorithm 1.

Table 1 Explanation of related symbols in the context-aware topic model CAT

In order to solve the parameters in the model, the present invention adopts the widely used Gibbs sampling (Gibbs sampling) algorithm.

First, according to the Gibbs sampling algorithm, the probability that the th word in the document is assigned to the topic is expressed by the following formula (1):

in, Represents the topic assignment vectors of all other words excluding the current i-th word, Indicates the number of words (excluding the current word) assigned to topic k in document d, Indicates the number of times word w _i is assigned to topic k in the entire corpus (excluding the current word). For missing superscripts or subscripts in symbolic representations (such as and ) means to sum the missing dimension, and 1 is a vector with all 1 elements.

Similarly, the probability that the j-th query-related context window in document d is assigned to topic k can be expressed by the following formula (2):

in, Indicates the topic assignment vector for all other windows excluding the current j-th context window relevant to the query, Indicates the number of all context windows (excluding the current window) related to query Q in topic k, θ _d,k represents the probability of topic k in document d, which can be further calculated by the following formula:

${\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; no</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}^{\mathrm{<span\; class="notranslate">\; d</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}}{{\mathrm{<span\; class="notranslate">\; no</span>}}_{<span\; class="notranslate">\; (</span><span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; d</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}^{\mathrm{<span\; class="notranslate">\; T</span>}}\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>$

in, Indicates the total number of words in document d that are assigned to topic k.

When the model converges or reaches the preset number of iterations, the following distributions will be obtained: "document-topic" distribution θ, "topic-word" distribution Φ and "topic-query context" distribution η. Each column of η represents the distribution of all relevant contexts for a certain query over topics, which is also the resulting new query representation. It can be seen that the representation naturally fuses contextual information and topic information at the same time, which is theoretically superior to representations that model each separately.

(3) Hybrid retrieval model design

Based on the obtained new query representation, the present invention designs a hybrid retrieval model that simultaneously considers keyword matching and topic matching, and its retrieval score calculation formula is as follows:

$\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; \&lambda;</span>}<span\; class="notranslate">\; )</span>{<span\; class="notranslate">\; \&Sigma;</span>}_{{\mathrm{<span\; class="notranslate">\; q</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; \&Element;</span>\mathrm{<span\; class="notranslate">\; Q</span>}}\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; q</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; d</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&lambda;</span>}<span\; class="notranslate">\; \&CenterDot;</span>{\mathrm{<span\; class="notranslate">\; the\; s</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; Q</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; d</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span>$

Where s represents the score based on keyword matching in the traditional retrieval model, such as language model retrieval score or BM25 retrieval score, s′ represents the topic matching score based on the new query representation Q′, and λ is the weight parameter between these two scores, It is also the trade-off coefficient of the two matching methods.

Regarding the topic matching score, various calculation methods can be adopted. Specifically, given the new query representation and the topic distribution vector of the document, it can be obtained by calculating the topic distribution similarity between the two, such as Jensen-Shannon divergence (JSD) and cosine similarity (Cosine similarity).

The protection content of the present invention is not limited to the above embodiments. Without departing from the spirit and scope of the inventive concept, changes and advantages conceivable by those skilled in the art are all included in the present invention, and the appended claims are the protection scope.

Claims (6)

1. A query representation based on context-aware topics and a method for establishing a hybrid retrieval model, characterized in that, comprising the steps of: Step 1: Obtain a pseudo-relevant feedback document of the query based on the keyword set of the query, and select a context related to the query from the pseudo-relevant feedback document; Step 2: introducing a context-aware topic model, integrating the context into the context-aware topic model, mining the topic information implied by the context window based on the corpus topic, and obtaining its corresponding topic vector; Step 3: The query is jointly represented by the topic vector and the keyword set, and based on the topic vector and the keyword set, a hybrid retrieval model is established to obtain a final retrieval score. 2. The query representation and hybrid retrieval model building method based on context-aware topics as claimed in claim 1, wherein in step 1, the pseudo-relevant feedback documents are divided into a plurality of sliding windows, and each window is calculated For the correlation with the query, take the window whose correlation is higher than the threshold as the context window related to the query. 3. The context-aware topic-based query representation and hybrid retrieval model building method according to claim 2, wherein the context selection threshold related to the query is an average value of the correlations of all windows under the query. 4. The query representation and hybrid retrieval model building method based on context-aware topics as claimed in claim 1, wherein the context-aware topic model is designed according to the relevant context of the query and the entire corpus, utilizing the context-aware topics In the process of topic modeling, the model assumes that the context window and the pseudo-relevant feedback document in which it is located share the same topic distribution, and the topic vector of the context is obtained. 5. The context-aware topic-based query representation and hybrid retrieval model building method according to claim 1, wherein the pseudo-relevant feedback document is obtained by calculating the keyword matching score of the retrieval model. 6. The query representation and hybrid retrieval model building method based on context-aware topics as claimed in claim 1, wherein the retrieval score is represented by the following formula: $\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; \&lambda;</span>}<span\; class="notranslate">\; )</span>{<span\; class="notranslate">\; \&Sigma;</span>}_{{\mathrm{<span\; class="notranslate">\; q</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; \&Element;</span>\mathrm{<span\; class="notranslate">\; Q</span>}}\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; q</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; d</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&lambda;</span>}<span\; class="notranslate">\; \&Center\; Dot;</span>{\mathrm{<span\; class="notranslate">\; the\; s</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; Q</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; d</span>}<span\; class="notranslate">\; )</span>$ Among them, s represents the score based on keyword matching in the traditional retrieval model, s' represents the topic matching score based on the new query representation Q', and λ is the weight parameter between these two scores, and it is also the trade-off coefficient of the two matching methods.