CN114048386A - Search method integrating personalization and diversification based on deep learning - Google Patents

Search method integrating personalization and diversification based on deep learning Download PDF

Info

Publication number
CN114048386A
CN114048386A CN202111391539.0A CN202111391539A CN114048386A CN 114048386 A CN114048386 A CN 114048386A CN 202111391539 A CN202111391539 A CN 202111391539A CN 114048386 A CN114048386 A CN 114048386A
Authority
CN
China
Prior art keywords
query
user
diversity
document
representing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202111391539.0A
Other languages
Chinese (zh)
Other versions
CN114048386B (en
Inventor
窦志成
王淑婷
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Renmin University of China
Original Assignee
Renmin University of China
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Renmin University of China filed Critical Renmin University of China
Priority to CN202111391539.0A priority Critical patent/CN114048386B/en
Publication of CN114048386A publication Critical patent/CN114048386A/en
Application granted granted Critical
Publication of CN114048386B publication Critical patent/CN114048386B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/953—Querying, e.g. by the use of web search engines
    • G06F16/9535—Search customisation based on user profiles and personalisation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

The invention realizes a search method integrating individuation and diversification based on deep learning by a method in the field of intelligent search. The method mainly comprises three steps of calculating the weight of general diversity and personalized diversity, dynamically modeling the diversity and aggregating each score result, a search model (DFSPD model) which is based on deep learning and integrates the personalization and the diversity is taken as the core of the method, the diversity is divided into the personalized diversity and the general diversity to simultaneously consider the influence of the personalization and the diversity on the result, and the result which is more satisfactory to users is provided according to different conditions. The method and the device have the advantages that the flexibility is very high, the user description portrait generator, the word embedding matrix and the personalized weighting process can be replaced, and the accuracy is greatly improved.

Description

Search method integrating personalization and diversification based on deep learning
Technical Field
The invention relates to the technical field of network search, in particular to a search method fusing personalization and diversification based on deep learning.
Background
Ambiguity of a search statement is an important factor influencing the satisfaction degree of a search engine result, and two main ambiguity removing methods are available, namely personalized search and diversification of the search result. Personalized searches typically build models that can learn information about user interests and preferences from a user's search history to clarify the user's specific intent in the current situation, thereby providing documents that match the user's interests. From early personalized learning based on artificial features, such as click features, word frequency features and the like, to learning of more complex and abstract correlation features based on various deep learning models, such as RNN, GAN, Transformer and the like, personalized search is always dedicated to mining more accurate and fine-grained interest features of a user, but the problems of redundancy of results, incomplete capture of user interest and the like are caused by neglecting the diversity of the results and the inherent diversity of the user interest, and meanwhile, when the user does not have historical data, the user cannot express good performance, namely the problem of 'cold start'. The diversification of search results is expected to ensure that the model can return sufficiently diverse document sets to meet different intentions of different users as much as possible on the premise of ensuring the content relevance, so as to achieve the purpose of disambiguation. Early diversification models typically designed fixed formulas that calculated diversity based on content similarity or subtopic coverage, so that locally optimal documents were greedy selected to compose the returned results. The development of machine learning enables diversified models to learn semantically based features to more reasonably compute diversity. The diversification is limited in that specific intentions of the user cannot be accurately extracted, so that an optimal document ranking result cannot be generated, and a large number of documents which are not interested by the current user are contained in a search result. The algorithm combining individuation and diversification can combine the advantages of the two algorithms to realize the complementation of the advantages and the disadvantages, thereby providing more satisfactory search results for users. At present, research in the field is deficient, personalized diversification of most models is realized by introducing personalized factors on a traditional diversification algorithm, and personalized and diversified static fusion work is realized by a structured SVM model in the near future.
The existing fusion work of individuation and diversification mainly has the following problems:
(1) under different conditions, the emphasis of personalization and diversification should be different, for example, when a new user uses a search engine, or the user searches for a problem that the difference from the history is large, and the model cannot learn useful information from the search history, the emphasis should be placed on diversification to improve the probability of user satisfaction, conversely, the emphasis should be placed on personalization, most of the existing methods only simply introduce a personalization factor of user information to improve the diversified model, or statically consider the fusion of personalization and diversification, and do not dynamically consider the weighting problem of the two.
(2) The interest of the user has inherent diversity, so the personalization is emphasized, the diversity of the user intention is emphasized, and the incomplete capture of the user intention is prevented.
(3) The model based on deep learning can learn feature information on high-order semantics, and most of the existing fusion models are based on manual features or a traditional machine learning method, so that the accuracy is greatly improved.
Disclosure of Invention
Therefore, the invention firstly provides a search method integrating individuation and diversification based on deep learning, historical query data, current query and candidate document set of a user are input, and the search method is realized through three steps: the method comprises the following steps: based on HRNN model according to long-term history H of userlShort term history HsLearn its initial long-term and short-term description picture L0And S0In which H isl＝{{q1,D1},…,{qn,DnH, n represents the total number of queries H contained in the previous sessions＝{{qn+1,Dn+1},…,{qn+m,Dn+mJ, m is the number of queries that have been searched in the current session, qnRepresenting the nth query, D, made by the usernFor querying qnReturning the initial candidate document set by the time ad-hoc search engine, further using the matching degree of the query vector and the user description image, and using div (d | q)v) Representing general diversity of document d, div (d | S), div (d | L) representing personalized diversity of documents, using rS,rLDynamically controlling the influence degree of the personalized diversity on the scoring result as the weight of the personalized diversity; step two: constructing an RRNN model, selecting a local optimal document d of each step based on the current comprehensive score (d) of the documents in the residual candidate document set by utilizing a greedy algorithm, and then selecting the local optimal document d according to the current comprehensive scoreThe virtual subtopic representation of the document is used to learn the influence of the document on the subtopic representation of the user long and short term description portraits and queries, the reset gate structure is used to model the influence, and the updating is carried out
Figure BDA0003368696580000021
Is composed of
Figure BDA0003368696580000022
Wherein
Figure BDA0003368696580000023
And after the sub-topics covered by the t selected documents are forgotten to be respectively expressed, the vector representations of the query sentence, the user short-term portrait and the user long-term portrait are used for calculating the score of the next document selection.
Figure BDA0003368696580000024
Representing the initiability of the query and user portrait vector representations. Finally, dynamic modeling of modeling document diversity is realized; and step three, aggregating all score results, and obtaining the basis for reordering the candidate documents based on the scores, thereby realizing the search result reordering algorithm integrating individuation and diversification. The document total score function is as follows:
score(d)＝P(d|q,D,S,u)＝P(d|qv,S,L)＝
φ(div(d|qv),rSdiv(d|S),rLdiv(d|L),rel(d,q))
wherein div (d | q)v) Representing general diversity of document d, div (d | S), div (d | L) representing personalized diversity of documents, using rS,rLAnd as the weight of the personalized diversity to dynamically control the influence degree of the personalized diversity on the score, rel (d, q) represents the correlation score of the query and the document obtained by calculation based on additional manual features, and phi is an aggregation function and is realized by using a multilayer perceptron.
The specific calculation mode of the general diversity weight and the personalized diversity weight is as follows: word embedding is learned using word2vec techniques, and then word-based TF-IDF weightingVector representation by weighted summation
Figure BDA0003368696580000031
Learning user initial long-short term description portrait H 'by using HRNN structure'0,S'0；
The HRNN structure firstly uses a first-layer RNN to take all query records of users as input, and each session is ended to construct interest characterization of the users in each session for period nodes:
Figure BDA0003368696580000032
subscripts m and n denote the nth query record in the mth session, superscript 1 denotes the first-tier RNN structure, qm,n,dm,nRepresenting the corresponding input query and the average relevant documents, the user's short-term description representation may be represented as:
Figure BDA0003368696580000033
where M denotes the Mth current session, nMIndicating that a search has been conducted by the user in the session;
the long-term description sketch is constructed according to the whole history record which does not contain the current session record, firstly, each session representation of the user history is used as input, and a second-layer RNN structure is applied to learn the interest representation of each period of the user:
Figure BDA0003368696580000034
where m denotes the mth current session, nmIndicating that the user has performed a search in the session, and then utilizes the current query
Figure BDA0003368696580000035
And user interest characterization for each period
Figure BDA0003368696580000036
Degree of similarity of (a)mAnd as the weights of the interests of the various periods, taking the result of weighted summation as the long-term interest description of the user:
Figure BDA0003368696580000037
Figure BDA0003368696580000038
Figure BDA0003368696580000039
introducing a convolutional neural network conv to learn a virtual sub-topic representation sequence thereof from the original representation of the query:
Figure BDA0003368696580000041
wherein q isiRepresents the ith sub-topic characterization of the query, and c represents the number of sub-topics contained by the query. Similar sub-topic representations of documents and user long-short term description portraits can be obtained: d, L0,S0：
d＝[d1,d2…dc]＝conv(d')
L0＝[L1,L2…Lc]＝conv(L'0)
S0＝[S1,S2…Sc]＝conv(S'0)
Furthermore, a KNRM model was introduced, using tq＝[tq1,...,tqn],td＝[td1,...,tdm]To represent the word sequences of the query and the word sequences of the document, where tqiThe ith word-embedding vector, t, representing the querydjA j word embedding vector representing the document, n and m respectively representing the number of words of the current query and document, and KNRM constructing tran according to cosine similarity between each group of word pairs of the query and documentslop matrix M:
Mij＝cos_similarity(tqi,tdj)
Figure BDA0003368696580000042
then the KNRM learns the multi-dimensional characteristics on a translation matrix by using k Gaussian kernels, and then the similarity of the query and the document is calculated by using a multilayer perceptron, wherein the specific calculation steps are as follows:
FK(tq,td)＝φ(f1(M),...fo(M),...,fk(M))
Figure BDA0003368696580000043
wherein u iso,σoThe mean and standard deviation corresponding to the o-th gaussian kernel are manually set hyper-parameters, and then the weighted value can be expressed as:
Figure BDA0003368696580000044
the reset gate is constructed in the following manner:
Figure BDA0003368696580000045
wherein the content of the first and second substances,
Figure BDA0003368696580000046
show that
Figure BDA0003368696580000047
Spliced together, WqIs a learnable mapping parameter, will
Figure BDA0003368696580000048
The mapping is a real number, σ (·) representing the activation function.
The activation function is a tanh function.
The update function of the reset gate is:
Figure BDA0003368696580000049
the total score is calculated using a learnable matrix WaAggregating the individual diversity and relevance scores to yield:
score(d)＝φ(s)＝tanh(sTWa)
where s denotes a vector formed by the respective sub-scores, phi denotes a multi-layer perceptron with tanh as an activation function, and s ═ div (d | q)v),rSdiv(d|S),rLdiv(d|L),rel(d,q)]Wherein q isvS, L are
Figure BDA0003368696580000051
Omitting the superscript form of t, d, q of the latest featurevS, L both represent corresponding virtual sub-topic sequences, and the scores of general diversity and personalized diversity are computed using the KNRM structure:
div(d|qv)＝FK(d,qv)
div(d|S)＝FK(d,S)
div(d|L)＝FK(d,L)
rel(d,q)＝φ(f(d,q))
additional manual feature sequences for documents and queries are represented using f (d, q), with the individual manual features being fused using a multi-tier perceptron φ.
The loss function of the method is constructed by using a two-classification loss function:
Figure BDA0003368696580000052
where Q represents all queries in the training set, QoRepresenting all sample pairs, y, to which the query q correspondsoIs a sample label when
Figure BDA0003368696580000053
Ratio of
Figure BDA0003368696580000054
When the user demand can be satisfied, y o1, and conversely 0,
Figure BDA0003368696580000055
representing model predictions
Figure BDA0003368696580000056
Ratio of
Figure BDA0003368696580000057
Better probability, the resample pairs are written in the form of a pair: (C, d)1,d2) Is calculated by the following formula
Figure BDA0003368696580000058
Figure BDA0003368696580000059
Finally, gradually optimizing the model by an AdamaOptimizer optimizer.
The technical effects to be realized by the invention are as follows:
a search method is realized and has the following characteristics:
(1) and dynamically learning personalized and diversified weights according to the matching degree of the query proposed by the user and the user description portrait.
(2) Diversity is divided into personalized diversity, which captures the inherent diversity of user interests, and general diversity, which deals with cold start and the problem of useless historical information.
(3) Virtual subtopic representations of learnable users, query sentences and documents are constructed, so that the displayed subtopic representations which take the diversity among the documents into consideration are learnt, and the subtopic representations with wider coverage are learnt.
(4) And dynamically modeling the diversity of the documents by utilizing a neural network, and constructing personalized and diversified branches according to the long-term and short-term description portrait of the user and the query statement.
Drawings
FIG. 1 is a search model architecture based on deep learning that incorporates personalization and diversification;
FIG. 2 is a graphical representation of a reset gate structure
Detailed Description
The following is a preferred embodiment of the present invention and is further described with reference to the accompanying drawings, but the present invention is not limited to this embodiment.
The invention provides a search method fusing individuation and diversification based on deep learning. The method is based on a Search model fusing Personalization and diversification based on deep learning, and a DFSPD (dynamic Fusion Framework of Search Result Personalization and diversification) model is used as the core of the method, and the method divides the diversity into personalized diversity and general diversity to simultaneously consider the influence of the Personalization and the diversification on the Result. Specifically, a long-term and short-term description image of a user is learned based on the HRNN model, personalized and diversified weights are learned by utilizing the matching degree of a query vector and the user description image, and the RRNN model is constructed to dynamically model the diversity of documents. And introducing virtual subtopics to display and consider subtopic distribution, calculating the similarity of subtopic sequences by utilizing a similar KNRM structure, and finally fusing each sub-score by using a multilayer perceptron.
Suppose that for a user u, its history data contains a long-term history HlAnd short term history HsThe former including the interaction behavior H in the previous sessionl＝{{q1,D1},…,{qn,DnH, where n represents the total number of queries contained in the previous session, the latter containing a list of queries in the current session and candidate document Hs＝{{qn+1,Dn+1},…,{qn+m,Dn+mAnd m is the number of queries that have been searched in the current session. The DFSPD model is firstly based on the HRNN model according to the long-term history H of the userlShort term history HsLearn its initial long-term and short-term description picture L0And S0. When the user submits a query q, the ad-hoc search engine returns an initial set of candidate documents D ═ D { (D)1,d2,.. }, our model would select the locally optimal document d for each step greedy based on the document's current composite score (d)*. After the local optimal document is selected, the model learns the influence of the document on the sub-topic characterization of the long-short term description portrait of the user and the query according to the virtual sub-topic characterization of the document, models the influence by using the reset gate structure designed by the user, and updates the model
Figure BDA0003368696580000071
Is composed of
Figure BDA0003368696580000072
For convenience, we will hereinafter sometimes refer to
Figure BDA0003368696580000073
Reduced to qvS, L, we use
Figure BDA0003368696580000074
To emphasize the initiatives of query and user portrait vector representations, i.e., the state when no documents are selected. The document total score function is as follows:
score(d)＝P(d|q,D,S,u)＝P(d|qv,S,L)＝φ(div(d|qv),rSdiv(d|S),rLdiv(d|L),rel(d,q))
wherein div (d | q)v) Representing the general diversity of the document d, div (d | S), div (d | L) representing the personalized diversity of the document, we use rS,rLAnd dynamically controlling the influence degree of the personalized diversity on the score as the weight of the personalized diversity. rel (d, q) represents the relevance score of the query and document computed based on the additional manual features. φ is an aggregation function, which we implement using multi-layer perceptrons (MLPs). The model structure is shown in fig. 1.
The inputs to the model include the user's historical query data, the current query, and the set of candidate documents. The model mainly comprises the following steps:
1. weights for general diversity and personalized diversity are calculated.
2. Dynamic modeling of diversity.
3. Aggregating the individual scoring results.
Calculating weights for general diversity and personalized diversity:
as we have said earlier, personalization and diversification should be given different weights based on different situations, and more specifically, such weights are learned based on the similarity of the current query and the user description representation. Meanwhile, a convolutional neural network is utilized to learn virtual subtopic representation of user figures, queries and documents, and therefore matching degree between the user figures, the queries and the documents is considered according to the subtopic.
For modeling of documents and query sentences, we use word2vec techniques to learn word embedding, and then get their vector representation based on a weighted summation of the words' TF-IDF weights
Figure BDA0003368696580000075
We learn the user's initial long-short term description portrait H ' using the HRNN structure '0,S'0。
The idea of the HRNN architecture is to utilize a hierarchical RNN model to build a long-short term description sketch of a user from the user's search history. Firstly, a first-layer RNN is utilized to take all query records of users as input, and each session is ended to construct interest characterization of the users in each session for a period node:
Figure BDA0003368696580000076
subscripts m and n denote the nth query record in the mth session, superscript 1 denotes the first-tier RNN structure, qm,n,dm,nRepresenting the corresponding input query and the average relevant documents. Let the current session be the Mth session in which the user has already made nMThe secondary search, the user's short-term representation may be represented as:
Figure BDA0003368696580000081
the long-term description sketch is constructed according to the overall history record (not including the current session record), firstly, each session representation of the user history is taken as input, and a second-layer RNN structure is applied to learn the interest representation of each period of the user:
Figure BDA0003368696580000082
let us assume that the user has performed n in the m-th sessionmAnd (5) secondary searching. Thereafter utilizing the current query
Figure BDA0003368696580000083
And user interest characterization for each period
Figure BDA0003368696580000084
Degree of similarity of (a)mAnd as the weights of the interests of the various periods, taking the result of weighted summation as the long-term interest description of the user:
Figure BDA0003368696580000085
Figure BDA0003368696580000086
Figure BDA0003368696580000087
in order to obtain a fine-grained sub-topic with a wider coverage, a convolutional neural network conv is introduced to learn a virtual sub-topic representation sequence from an original representation of a query:
Figure BDA0003368696580000088
wherein q isiRepresents the ith sub-topic characterization of the query, and c represents the number of sub-topics contained by the query. Similar sub-topics capable of obtaining long-short term description portrait of document and userRepresents: d, L0,S0。
To enable learning of finer grains from multiple dimensions
Figure BDA0003368696580000089
And S0,L0We introduced the KNRM model. KNRM is a common ad-hoc model, i.e. document ranking is based only on content similarity, regardless of personalization or diversification. The input to a conventional KNRM is a query and a document, both of which are represented by corresponding word-embedded vector sequences. We use tq＝[tq1,...,tqn],td＝[td1,...,tdm]To represent the word sequences of the query and the word sequences of the document, where tqiThe ith word-embedding vector, t, representing the querydjThe jth word representing a document is embedded into the vector, and n and m represent the number of words of the current query and document, respectively. The KNRM constructs a translation matrix M according to cosine similarity between each group of word pairs of the query and the document:
Mij＝cos_similarity(tqi,tdj)
Figure BDA0003368696580000091
then the KNRM learns the multi-dimensional characteristics on a translation matrix by using k Gaussian kernels, and then the similarity of the query and the document is calculated by using a multilayer perceptron (MLP), wherein the specific calculation steps are as follows:
FK(tq,td)＝φ(f1(M),...fo(M),...,fk(M))
Figure BDA0003368696580000092
wherein u iso,σoThe mean and standard deviation corresponding to the o-th gaussian kernel are shown as manually set hyper-parameters. We apply the idea of KNRM similarity to compute the similarity of sub-topic sequencesIt is possible to obtain:
Figure BDA0003368696580000093
dynamic modeling of diversity:
when the current locally optimal document d is selected*Later, we want the query and user profile to be forgotten and d*Related aspects, whereby in the following document selection, emphasis is placed on d*The angle that has not been covered yet. Based on this assumption, we build a Reset gate (Reset gate):
Figure BDA0003368696580000094
wherein the content of the first and second substances,
Figure BDA0003368696580000095
show that
Figure BDA0003368696580000096
Spliced together, WqIs a learnable mapping parameter, will
Figure BDA0003368696580000097
The mapping is a real number. σ (-) denotes the activation function, and the tanh function is used herein.
The structure is graphically represented in figure 2.
The use of a reset gate alone has several drawbacks: 1. the candidate documents are usually dozens, the tanh function is a function for mapping real numbers to (-1,1), and the multiplication of a plurality of numbers with absolute values smaller than 1 results in a value with small absolute values, which may make the vector lose expression capability at a later stage and also cause problems such as gradient disappearance. 2. After browsing a certain number of documents, the user may forget what was seen before and may therefore want to click on a document similar to the previously browsed document. Inspired by resnet, our update function is as follows:
Figure BDA0003368696580000098
the structure not only can well solve the problem of gradient disappearance, but also gives the model an opportunity to remember the prior feature distribution. S, St,LtThe same updating method is also used. We refer to this RNN structure as a unit of reset gates as RRNN.
Aggregating the individual score results:
mmr (maximum geographic relevance) is a classical structure that balances the relevance score and diversity score. He linearly combines the correlation score and the diversity score by introducing a balance factor λ. However, λ is a manually set hyper-parameter and is in most cases not an optimal choice. To find a more appropriate balance factor, we use a learnable matrix WaAggregating the individual diversity and relevance scores into a final score:
score(d)＝φ(s)＝tanh(sTWa)
where s represents a vector formed by the sub-scores, and phi represents a multi-layer perceptron with tanh as an activation function. s ═ div (d | q)v),rSdiv(d|S),rLdiv(d|L),rel(d,q)]Here, for the sake of brevity, we omit the t superscript denoting the latest features, where d, qvS, L both represent the corresponding virtual sub-topic sequence. Wherein, the scores of general diversity and personalized diversity are calculated by using KNRM structure:
div(d|qv)＝FK(d,qv)
div(d|S)＝FK(d,S)
div(d|L)＝FK(d,L)
rel(d,q)＝φ(f(d,q))
we use f (d, q) to represent additional manual feature sequences for documents and queries, and use a multi-tier perceptron φ to fuse individual manual features, computing relevance scores.
Training and optimizing:
in previous personalization work, researchers thought to be clicked onThe documents of (2) reflect only the relevance of the documents to the user and the query, however, the behavior of the user is complex, and a redundant relevant document may not be clicked on by the user because the information requirements of the user in this respect have been met. That is, the user's behavior reflects not only document relevance, but also document diversity. Therefore, the document clicked by the user at each step is regarded as a positive sample meeting the requirements of the user from the relevance and diversity, and is regarded as a negative sample in the opposite. One sample pair of our model can be represented as (r)1,r2)，r1,r2Representing two document ranks, our model is apparently the List-pair model, however the difference between these two ranks is only reflected in the last document, so this sample pair can also be written as (C, d)1,d2) Where C represents a set of selected documents, d1,d2Represents the last sample in the ordering, let d1Is a positive sample, d2Is a negative example, so we can reduce our model to the C-based pairwise method and use the two-class loss function to construct the loss function of our model:
Figure BDA0003368696580000111
where Q represents all queries in the training set, QoRepresenting all sample pairs corresponding to query q. y isoIs a sample label when
Figure BDA0003368696580000112
Ratio of
Figure BDA0003368696580000113
When the user demand can be satisfied, y o1, otherwise 0,.
Figure BDA0003368696580000114
Representing model predictions
Figure BDA0003368696580000115
Ratio of
Figure BDA0003368696580000116
Better probability. Because our sample pairs can be written in the form of a pairwise: (C, d)1,d2) We can calculate by the following formula
Figure BDA0003368696580000117
Figure BDA0003368696580000118
Finally, gradually optimizing the model by an AdamaOptimizer optimizer.

Claims (6)

1. A search method fusing personalization and diversification based on deep learning is characterized in that: inputting historical query data, a current query and a candidate document set of a user, and realizing the method through three steps: the method comprises the following steps: based on HRNN model according to long-term history H of userlShort term history HsLearn its initial long-term and short-term description picture L0And S0In which H isl＝{{q1,D1},…,{qn,DnH, n represents the total number of queries H contained in the previous sessions＝{{qn+1,Dn+1},…,{qn+m,Dn+mJ, m is the number of queries that have been searched in the current session, qnRepresenting the nth query, D, made by the usernFor querying qnReturning the initial candidate document set by the time ad-hoc search engine, further using the matching degree of the query vector and the user description image, and using div (d | q)v) Representing general diversity of document d, div (d | S), div (d | L) representing personalized diversity of documents, using rS,rLDynamically controlling the influence degree of the personalized diversity on the score as the weight of the personalized diversity; step two: constructing an RRNN model, and selecting the local optimal document d of each step according to the current comprehensive score (d) of the candidate documents in the residual candidate document set by utilizing a greedy algorithm*Then according to the virtual sub-words of the documentTopic representations to learn the impact of the document on the user's long and short term description portraits and sub-topic representations of queries, using reset gate structures to model such impacts, updates
Figure FDA0003368696570000011
St-1,Lt-1Is composed of
Figure FDA0003368696570000012
St,LtWherein, in the step (A),
Figure FDA0003368696570000013
St,Ltrespectively representing vector representations of query sentences, user short-term portraits and long-term portraits after forgetting sub-topics covered by t selected documents for the score calculation of the next document selection,
Figure FDA0003368696570000014
L0,S0representing the initiability of the query and user portrait vector representations. Finally, dynamic modeling of modeling document diversity is realized; and step three, aggregating all score results, wherein the scores are the basis for reordering the candidate documents, and outputting the reordered search results.
2. The deep learning-based search method integrating personalization and diversification as claimed in claim 1, wherein: the specific calculation mode of the general diversity weight and the personalized diversity weight is as follows: word embedding is learned using word2vec techniques, and then vector representation based on a weighted summation of word TF-IDF weights
Figure FDA0003368696570000015
d ', learning the initial long-short term description portrait H ' by HRNN structure '0,S'0；
The HRNN structure firstly uses a first-layer RNN to take all query records of users as input, and each session is ended to construct interest characterization of the users in each session for period nodes:
Figure FDA0003368696570000016
subscripts m and n denote the nth query record in the mth session, superscript 1 denotes the first-tier RNN structure, qm,n,dm,nRepresenting the corresponding input query and the average relevant documents, the user's short-term description representation may be represented as:
Figure FDA0003368696570000021
where M denotes the Mth current session, nMIndicating that a search has been conducted by the user in the session;
the long-term description sketch is constructed according to the whole history record which does not contain the current session record, firstly, each session representation of the user history is used as input, and a second-layer RNN structure is applied to learn the interest representation of each period of the user:
Figure FDA0003368696570000022
where m denotes the mth current session, nmIndicating that the user has performed a search in the session, and then utilizes the current query
Figure FDA0003368696570000023
And user interest characterization for each period
Figure FDA0003368696570000024
Degree of similarity of (a)mAnd as the weights of the interests of the various periods, taking the result of weighted summation as the long-term interest description of the user:
Figure FDA0003368696570000025
Figure FDA0003368696570000026
Figure FDA0003368696570000027
introducing a convolutional neural network conv to learn a virtual sub-topic representation sequence thereof from the original representation of the query:
Figure FDA0003368696570000028
wherein q isiThe representation of the ith sub-topic of the query, c represents the number of sub-topics contained in the query and further obtains the representation of the sub-topics of the document and the user long-short term description portrait: d, L0,S0:
d＝[d1,d2…dc]＝conv(d')
L0＝[L1,L2…Lc]＝conv(L'0)
S0＝[S1,S2…Sc]＝conv(S'0)
Furthermore, a KNRM model was introduced, using tq＝[tq1,...,tqn],td＝[td1,...,tdm]To represent the word sequences of the query and the word sequences of the document, where tqiThe ith word-embedding vector, t, representing the querydjA jth word embedding vector representing a document, n and M respectively represent the number of words of the current query and the document, and the KNRM constructs a translation matrix M according to cosine similarity between each group of word pairs of the query and the document:
Mij＝cos_similarity(tqi,tdj)
Figure FDA0003368696570000029
then the KNRM learns the multi-dimensional characteristics on a translation matrix by using k Gaussian kernels, and then the similarity of the query and the document is calculated by using a multilayer perceptron, wherein the specific calculation steps are as follows:
Fk(tq,td)＝φ(f1(M),...fo(M),...,fk(M))
Figure FDA0003368696570000031
wherein u iso,σoThe mean and standard deviation corresponding to the o-th gaussian kernel are manually set hyper-parameters, and then the weighted value can be expressed as:
Figure FDA0003368696570000032
3. the deep learning-based search method integrating personalization and diversification as claimed in claim 2, wherein: the reset gate is constructed in the following manner:
Figure FDA0003368696570000033
wherein the content of the first and second substances,
Figure FDA0003368696570000034
show that
Figure FDA0003368696570000035
d*Spliced together, WqIs a learnable mapping parameter, will
Figure FDA0003368696570000036
The mapping is a real number, σ (·) representing the activation function.
4. The deep learning-based search method integrating personalization and diversification as claimed in claim 3, wherein: the activation function is a tanh function.
5. The deep learning-based search method integrating personalization and diversification as claimed in claim 4, wherein: the update function of the reset gate is:
Figure FDA0003368696570000037
the total score is calculated using a learnable matrix WaAggregating the individual diversity and relevance scores to yield:
score(d)＝φ(s)＝tanh(sTWa)
where s denotes a vector formed by the respective sub-scores, phi denotes a multi-layer perceptron with tanh as an activation function, and s ═ div (d | q)v),rSdiv(d|S),rLdiv(d|L),rel(d,q)]Wherein q isvS, L are
Figure FDA0003368696570000038
St,LtOmitting the superscript form of t, d, q of the latest featurevS, L both represent corresponding virtual sub-topic sequences, and the scores of general diversity and personalized diversity are computed using the KNRM structure:
div(d|qv)＝FK(d,qv)
div(d|S)＝FK(d,S)
div(d|L)＝FK(d,L)
rel(d,q)＝φ(f(d,q))
f (d, q) is used to represent additional sequences of manual features about the document and the query, a multi-tier perceptron phi is used to fuse the individual manual features, and rel (d, q) represents the relevance score of the query and document computed based on the additional manual features.
6. The deep learning-based search method integrating personalization and diversification as claimed in claim 5, wherein: the loss function of the method is constructed by using a two-classification loss function:
Figure FDA0003368696570000041
where Q represents all queries in the training set, QoRepresenting all sample pairs, y, to which the query q correspondsoIs a sample label when
Figure FDA0003368696570000042
Ratio of
Figure FDA0003368696570000043
When the user demand can be satisfied, yo1, and conversely 0,
Figure FDA0003368696570000044
representing model predictions
Figure FDA0003368696570000045
Ratio of
Figure FDA0003368696570000046
Better probability, the resample pairs are written in the form of a pair: (C, d)1,d2) Is calculated by the following formula
Figure FDA0003368696570000047
Figure FDA0003368696570000048
Finally, gradually optimizing the model by an AdamaOptimizer optimizer.
CN202111391539.0A 2021-11-23 2021-11-23 Deep learning-based personalized and diversified search method Active CN114048386B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111391539.0A CN114048386B (en) 2021-11-23 2021-11-23 Deep learning-based personalized and diversified search method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111391539.0A CN114048386B (en) 2021-11-23 2021-11-23 Deep learning-based personalized and diversified search method

Publications (2)

Publication Number Publication Date
CN114048386A true CN114048386A (en) 2022-02-15
CN114048386B CN114048386B (en) 2024-04-09

Family

ID=80211164

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111391539.0A Active CN114048386B (en) 2021-11-23 2021-11-23 Deep learning-based personalized and diversified search method

Country Status (1)

Country Link
CN (1) CN114048386B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023243796A1 (en) * 2022-06-17 2023-12-21 Samsung Electronics Co., Ltd. Method and system for personalising machine learning models

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20110023304A (en) * 2009-08-31 2011-03-08 서울시립대학교 산학협력단 Method and system of configuring user profile based on a concept network and personalized query expansion system using the same
CN105068661A (en) * 2015-09-07 2015-11-18 百度在线网络技术（北京）有限公司 Man-machine interaction method and system based on artificial intelligence
US20190188295A1 (en) * 2017-12-15 2019-06-20 Accenture Global Solutions Limited Cognitive searches based on deep-learning neural networks
CN111241407A (en) * 2020-01-21 2020-06-05 中国人民大学 Personalized search method based on reinforcement learning
CN112182387A (en) * 2020-09-29 2021-01-05 中国人民大学 Personalized search algorithm for enhancing time information
CN112182373A (en) * 2020-09-25 2021-01-05 中国人民大学 Context expression learning-based personalized search method

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20110023304A (en) * 2009-08-31 2011-03-08 서울시립대학교 산학협력단 Method and system of configuring user profile based on a concept network and personalized query expansion system using the same
CN105068661A (en) * 2015-09-07 2015-11-18 百度在线网络技术（北京）有限公司 Man-machine interaction method and system based on artificial intelligence
US20190188295A1 (en) * 2017-12-15 2019-06-20 Accenture Global Solutions Limited Cognitive searches based on deep-learning neural networks
CN111241407A (en) * 2020-01-21 2020-06-05 中国人民大学 Personalized search method based on reinforcement learning
CN112182373A (en) * 2020-09-25 2021-01-05 中国人民大学 Context expression learning-based personalized search method
CN112182387A (en) * 2020-09-29 2021-01-05 中国人民大学 Personalized search algorithm for enhancing time information

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
官蕊;丁家满;贾连印;游进国;姜瑛;: "基于强化学习的多样性文档排序算法", 计算机工程与科学, no. 09, 15 September 2020 (2020-09-15) *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023243796A1 (en) * 2022-06-17 2023-12-21 Samsung Electronics Co., Ltd. Method and system for personalising machine learning models

Also Published As

Publication number Publication date
CN114048386B (en) 2024-04-09

Similar Documents

Publication Publication Date Title
CN109299396B (en) Convolutional neural network collaborative filtering recommendation method and system fusing attention model
CN109829104B (en) Semantic similarity based pseudo-correlation feedback model information retrieval method and system
US5619709A (en) System and method of context vector generation and retrieval
CN105659225B (en) Use the query expansion and inquiry-document matches of path constrained random migration
CN111581510A (en) Shared content processing method and device, computer equipment and storage medium
CN110555112A (en) interest point recommendation method based on user positive and negative preference learning
JP2020510910A (en) Machine learning method and apparatus for ranking network nodes after using a network with software agents at network nodes
León et al. Applying logistic regression to relevance feedback in image retrieval systems
US20190286978A1 (en) Using natural language processing and deep learning for mapping any schema data to a hierarchical standard data model (xdm)
El Mohadab et al. Predicting rank for scientific research papers using supervised learning
CN112182373B (en) Sexualization search method based on context representation learning
US20220172040A1 (en) Training a machine-learned model based on feedback
CN114358657B (en) Post recommendation method and device based on model fusion
CN112182387A (en) Personalized search algorithm for enhancing time information
CN108920521A (en) User's portrait-item recommendation system and method based on pseudo- ontology
CN116108267A (en) Recommendation method and related equipment
Ngueilbaye et al. SDLER: stacked dedupe learning for entity resolution in big data era
CN111651661A (en) Image-text cross-media retrieval method
Guo et al. Matching visual features to hierarchical semantic topics for image paragraph captioning
CN114048386A (en) Search method integrating personalization and diversification based on deep learning
Alper et al. Hybrid course recommendation system design for a real-time student automation application
CN113535949A (en) Multi-mode combined event detection method based on pictures and sentences
CN114036262B (en) Graph-based search result diversification method
Chang et al. Two-channel hierarchical attention mechanism model for short text classification
Xuan et al. Cooperative hierarchical Dirichlet processes: Superposition vs. maximization

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant